Broner very much picks his victims. What makes you think he'd dare to fight Matthysse?

Also, Broner gets hit pretty easily judging from the Ponce and Rees fights. That greatly favors Matthysse as it only takes one.

Broner is a lot of smoke and mirrors.