To be honest, not really. When it is cold or hot outside, solar panels prevent heat loss or cool down the building to some degree, but this effect is minor. If you better want to be safe than sorry, use the converted electricity from your solar panels for air-conditioning to heat or cool your building 😉.
It all depends on the building your solar panels are on. Several factors are important when you want to benefit from this side effect. Is there external insulation on the roof like EPS? Is the space well-ventilated? Is the space beneath the roof where the solar panels are on an empty space or an insulated work space?
Normally your house would capture the sun, but now the solar panels do. They are dark in color, gain but reflect heat, emit it and lose it by radiation and convection using the space between the roof on your building and the panels.
Photo by Greg Rosenke on Unsplash