Born in 1868 in Suzhou, his father was a constable in Suzhou before the family migrated to Shanghai to open a teahouse. During his childhood, Huang contracted a bad case of smallpox. While his subordinates called him 'Grand Master Huang', behind his back everyone called him 'Pockmarked Huang'. Huang went to work at his fathers teahouse, which was not very far from the Zhengjia Bridge near the French Concession. The bridge in those days sheltered a large population of hustlers and crooks. Huang Jinrong fitted right in, and organised many of them into a gang who later became his sworn followers. Aged 24, Huang passed the entrance exams and entered the French Concession police force, the Garde Municipale in 1892. Being strong, brash and capable, he did very well and became a detective in the Criminal Justice Section (Police Judiciaire). With the exception of a brief sojourn to Suzhou, Huang served continuously in the Police Judiciaire for twenty years until his retirement in 1925 after s