The performance of tritium-based nuclear batteries based on two different energy conversion modes, the irradiated voltaic effect and irradiated photovoltaic effect, was studied by using the Monte Carlo method. The influence of the geometrical-physical parameters of energy conversion materials on the electrical output performance of batteries wais investigated. Single-layer and stacked-layer tritium-based nuclear batteries were designed and prepared. The effects of increasing the tritium source intensity and adopting the stacked-layer configuration on the enhancement of the electrical output of the batteries were analyzed. The simulation results showed that Si, SiC, and GaAs photovoltaic modules could be used for irradiated voltaic effect tritium-based nuclear batteries and that their respective optimal thickness parameters allow the electrical output performance to be optimized; the optimal thicknesses were 3.8 μm, 2.2 μm, and 1.7 μm, respectively. For irradiated photovoltaic effect tritium-based nuclear batteries, the thickness of the ZnS:Cu fluorescent layer could be adjusted to maximize the emitted fluorescence irradiance and optimized the electrical output performance. The experimental results showed that increasing the radiation intensity of the tritium source and adopting the stacked-layer configuration could effectively enhance electrical parameters such as the maximum output power of tritium-based nuclear batteries. The maximum output power of the stacked-layer nuclear battery could reach 106.138 nW, which was an increase of more than 64% compared with that of the single-layer configuration.