It is said that Kam Fa Goddess was born to the family of a martial arts official. From a very young age she secretly learned kung fu from her father. Adapted with both pen and sword and characterized by a sense of justice, she robbed the rich to give to the poor and was worshipped as a goddess after her death.

More than 200 years ago, a herbal doctor came to Peng Chau to search for a remedy to save his beloved wife. He found a shrine for Kam Fa Goddess under a banyan tree so he knelt down to pray to the Goddess for a herbal remedy to cure her. Subsequently the medicine taken by his wife really produced a healing effect on her. To thank the Goddess the doctor went back to Peng Chau and built a Kam Fa Temple. Over time the temple was unattended and became dilapidated until the current carer Ms. KamLan, having been blessed by the Goddess' grace, restored it in 1978.

According to legend, Tin Hau Goddess was born in the Song Dynasty (960-1279A.C.) to a fisherman surnamed Lam in Fuzhou. As a child she was intelligent and knowledgeable in astrology and was able to forecast the climate. She also practiced medicine and saved countless people. After her death, she was worshiped as Tin Hau Goddess or Ma Jo Sea Goddess and had many followers around the South China Sea.

The older generation of Peng Chau residents owed much gratitude to Tin Hau Goddess for they survived many calamities under her protection.

Later Kam Fa Goddess found divine affinity with Master Kwan Tak Hing. In 1981 when Master Kwan was on a business trip in Japan, the Goddess revealed to him in a dream of her wish for him to go to Peng Chau to join the Kam Fa Festival - ceremony for her birth anniversary (the 17th day of the 4th month of the lunar calendar). There he would be received by the carer. On the other hand, the Goddess also instructed Ms. KamLan in a dream to welcome Master Kwan personally. Since then, during each year's Kam Fa Festival celebration Masters Kwan Tak Hing and Luk Chi Fu would lead about 10 lion-dancing teams to Peng Chau. Accordingly the temple gained popularity by attracting a large number of pilgrims. After his passing Master Kwan was conferred the title of Protection General to help Kam Fa Goddess to liberate all living beings from this mortal world.

Local legend maintains that Kam Fa Goddess is extremely accurate drawing worshippers from Hong Kong and everywhere. Peng Chau Kam Fa Temple is indeed unique in Hong Kong.