Could the inside of these walls cause an issue? (See details)

I’ll try to explain best I can. 1970’s mobile home. Panel walls, layer of insulation, and behind that is some kind of very thin wood board. That wood board is warped from mositure and crystallized. I’m not sure if that’s mold or what. The reason I know it looks this way is because I went into a closet (that has the back wall exposed to the outside) and pulled off a piece of the wall panel, pushed aside the insulation, and saw the warped wood board.

We are definitely aware this home has poor insulation. As a result if we put anything against a wall that’s other side is exposed to the outdoors, it will get moist from the cold and hot air mixing. Same with all of the windows. They’re all wet with condensation. We are in the process of having them all replaced.

The walls worry me. Could this be a major issues health wise? The first panel is fine and nothing is wet or crystallized on that. The insulation must protect it somewhat. But could that other wood board be something to be concerned about? Does crystallization on wood mean mold? If so, could that be getting through the other wall and hurting us?