If I understand your idea correctly, it's not gonna work, because it doesn't allow you to position the router table stock guide rollers in and out from the saw fence. The rollers are fixed in their distance from the fence, and can't handle wider stock. You can come up with a design that allows the rollers to be moved in and out, by adding another base (which is mounted on the saw fence with magswitches) for the L-fence to ride on.
One solution is to mount the Tablesaw Clear-Cut stock guide on a plywood base and lock the plywood base with two magswitches on the saw fence. The whole set-up isn't that heavy, and several members on this forum have used that method. Not a cheap solution, but as functional as screwing the guide on the sawfence.