This week I wanted to move past the triangle. My ultimate goal is to simulate a campus, and that triangle setup is fragile. It handles one downlink. Thats it. Worse, I thought about adding more routers until I realized — how would I even do that? There wasn’t an obvious path for scaleability. There are no new ideas, so I researched others who had the same problem.
Charles Clos was my guy. He said: divide the nodes into leaves and spines. Leaves connect to clients. Spines connect to leaves. And I can kill half the links and keep full connectivity1. But is full connectivity the goal?
To simulate a campus, each leaf is a classroom router. Then clients (e.g. student PCs) connect to their leaves. And I want those clients to talk to each other. I thought my network was perfect. Yet this setup allowed unregulated cross-classroom communication.
I optimized for a goal I didn’t want…
So I went back to the whiteboard. First off: classrooms (i.e. leaves) need more than one client. My existing single clients used point-to-point connections — why not add more? But this was an issue. They’re supposed to be part of the same classroom; why go through a mediator?2.
The solution was below me — at the link layer. If I let clients talk at the link layer, it avoided the mediation problem entirely. This is what bridges do. Clients talk through MAC addresses — students can ping each other without the router’s permission. Baked into this solution: external traffic still goes through the leaf. Perfect. I can regulate communication at the leaf level, and clients can chat without permission.
Now my network was starting to look like a school.
Footnotes
-
I could remove all but one spine and keep full redundancy. You can’t arbitrarily remove 50% of links ↩
-
Clients had to go through the leaf router to talk to each other. They’re point-to-point links, i.e. /31 pairs, and each one has a different subnet.
leaf1-client1 --> 192.168.1.2/31leaf2-client2 --> 192.168.1.4/31↩