In this sort of scenario it's common ground enough. Sam doesn't have the backing, the jurisdiction, and a heck of a lot bigger problems than The Punisher at the moment. If some of the other Avengers get here or that Devil of Hell's Kitchen, well, they can come back to this bridge and cross it. Hell, even if you didn't count Barnes they still had an ex-con on the team, even if Scott Lang was only put away for burglary.

"Sure. Glad we could have this chat," Sam says, because honestly what are you supposed to say?