Reddit reviews Stalag Wisconsin: Inside WWII Prisoner of War Camps

I can suggest one book I am familiar with on German and Japanese POWs interned in my home state of Wisconsin, which addresses parts of your question (on a limited scale as applicable to Wisconsin): Stalag Wisconsin, by Betty Cowley.

It covers the 38 POW camps (main camp and satellite branches) across WI, describing what prisoners did (e.g. many worked as laborers, just as you say, and were paid for their labor in small amounts; some few escaped, but never got far and often returned on their own), how some were moved south for the winter for other projects to work on, interactions with locals, etc. I cannot speak to how this specific case of POW camps in WI applies to the broader US and/or Canada.

You mentioning the POW camp reminded me of a great book on the subject... Stalag Wisconsin: Inside WWII Prisoner of War Camps. With Wisconsin's heavy German American influence the local interactions with escapees and prison laborers was kind of like a reverse Hogan's Hero's. I don't have the book in front of me but recall something about one German POW who would regularly "escape" the Mitchell Field camp to get a beer at a near-by tavern.