The laser interferometer and the optical encoder are commonly used for high-precision displacement measurement, which is significant for equipment manufacturing. The former can realize sub-micron measurement by counting and subdividing interference fringes. However, it has the disadvantages of strict requirements for the measurement environment and difficult integration directly into the equipment, which greatly limit its applications in industrial measurement and control. Compared with the laser interferometer, the encoder has been widely integrated with the CNC machine as a core measurement component due to its advantages of low cost, small size, and simple optical structure. Grating lithography has been successfully employed to fabricate the gratings of optical encoders. However, some inherent problems exist in this fabricating process, such as low production, long production cycle, and harsh production conditions. Furthermore, the optical encoder manufactured by the grating lithography requires a combination of a light source and a reading head. When the encoder has worked for a long period of time, the light source will dissipate a large amount of heat, resulting in a drastic change in the internal temperature of the encoder. The drastic temperature change will cause the thermal deformation error of the encoder substrate to influence its measurement precision. Due to the advantages of high efficiency, low cost, and simple process requirements, we have introduced additive manufacturing with perovskite quantum dots to fabricate a novel linear encoder named quantum dot encoder, which prints the perovskite quantum dot coding patterns on the substrate via additive manufacturing. Then, machine vision is applied to process the continuous, regular and winding quantum dot code pattern in real time to achieve displacement measurement. As a novel linear encoder, its measurement precision is significant for its wide applications. In order to further improve the measurement precision of the quantum dot encoder, a displacement measurement method based on triangular wave skeleton extraction of coding patterns is proposed in this paper. First, considering the winding and continuous shapes of coding patterns of the quantum dot encoder, the boundary tracking method with variable steps is proposed to detect the edges of coding patterns in real time. The detection path of this method is always kept around the edge of the coding patterns, so only a few pixels need to be traversed to detect the edges. Then, triangular wave fitting is carried out on the middle lines of coding patterns to obtain the triangular wave skeletons of coding patterns, so as to improve the measurement stability and the subdivision linearity of displacement. Finally, because of the advantages of simple structure, fast convergence, easy deployment, and good approximationperformance for nonlinear functions, a Radial Basis Function (RBF) neural network is used to compensate for the nonlinear errors emerging in the quantum dot encoder. A laser interferometer is used as a baseline for linear displacement measurement. We compared the three waveforms to fit the measured signal, which are triangle wave, sine wave and square wave. The experimental results show that the triangle wave can well fit the measured signal with a high amplitude and a low error rate. To validate the RBF neural network on the measurement error compensation of the quantum dot encoder, a BP network, an LSTM network and an RBF neural network are individually used to compensate for the measured displacement data while other experimental conditions remain unchanged. The experimental results show that the RBF neural network is superior to the other two neural networks in error compensation performance. To analyze the effect of three different steps in the proposed displacement measurement method, an ablation experiment is conducted. The experimental results show that the boundary tracking step with variable steps for coding pattern detection greatly accelerates the speed of boundary tracking. Compared with our previous work, the operation efficiency has increased by 108.86%. The coding pattern skeleton extraction method based on triangle wave fitting reduces the impact of environmental noise on the measurement precision, resulting in a reduction of the repetitive displacement error from ±8.695 μm to ±0.870 μm. The error compensation method based on the RBF neural network effectively compensates the nonlinear error of the quantum dot encoder, and improves the RMSE, maximum error, variance and confidence interval. Comparison experimental results indicate that the proposed method is more robust and achieves better measurement accuracy than the existing methods.