As China's largest CO2 emission agency, industrial carbon emission reduction is crucial to achieving China's CO2 emission reduction goals. Apart from technical upgrading and adjustment of energy consumption structure, industrial restructuring should also be one of the important measures to decrease carbon emission intensity. To understand the relationship between industrial structure and industrial carbon emission intensity, this paper analyzed the CKC relationship between the output value of each department and its CO2 emissions based on Chinese industry and energy data from 1986 to 2016 by classifying 41 industrial sectors into 16 departments. Then, the carbon emission intensity of total industry output was decomposed into the carbon emission department structure, carbon emission factor, energy consumption intensity, and output structure with Kaya function. Further, contributions of four factors to the change of Chinese industrial carbon emission intensity were studied with LMDI method. At last, some conclusions were drawn as follows. Firstly, the relationship between the output and its CO2 emissions of different industrial departments is different. Only the wood processing and furniture manufacturing, papermaking and printing, stationery manufacturing and non-metallic mineral products industries have an inverted U-shaped relationship. The department of machinery/transportation/electrical/electronic equipment manufacturing has an inverted N-shaped relationship, and the remaining departments have a linear or monotonous increasing relationship. Secondly, the decline of energy consumption intensity is the outstanding factor and all departments show the same trend, while the contributions of output structure and carbon emission department structure show different trends of different departments. The capital and technology-intensive industries, such as non-metallic mineral products, chemical industry, pharmaceutical industry, machinery/transportation/electrical/electronic equipment manufacturing, wood processing and furniture manufacturing, show the significant technical reduction effects. The primary product processing industries, such as other manufacturing, oil and gas extraction, textiles and apparel, and chemical fiber and rubber and plastics industries, have significant structure reduction effects, while some energy industries, such as petroleum processing, coke and nuclear fuel processing, metal smelting and processing, and electricity gas and water producing and supplying industry, show a strong size and structure effect simultaneously. Therefore, China needs to formulate a scientific and reasonable industrial structure adjustment policy based on the characteristics of the relationship between the output value of various industrial sectors and its CO2 emissions.