file transfer pseudoprotocol
file transfer pseudoprotocol
2020-05-20
Something I have been thinking about lately is FTP.
FTP is interesting in that it is one of the remnants of an earlier day of<br>internet protocol design. You can tell this, right off the bat, because it has<br>a low, odd port number. Odd-numbered ports in the low end are, broad speaking,<br>direct carry-overs from a pre-TCP protocol called NCP or Network Control<br>Program. Much like TCP, NCP was connection-oriented. Entirely unlike TCP, NCP<br>connections were one-way (simplex). So, a normal two-way connection between two<br>hosts required the establishment of two connections, one for each direction of<br>the dialog. To facilitate these two connections, protocols were each allocated<br>two port numbers for use with NCP, one for server-to-client and the other for<br>client-to- server. These ports were assigned adjacent and, by convention, an<br>odd-number port was allocated for client-to-server and the following<br>even-numbered port for server-to-client.
Part of what's going on here is that NCP predates the invention of the<br>'ephemeral port,' TCP's magical made up numbers. Both ends of a given NCP<br>connection make use of the same port number. So the client talks one one port<br>and listens on the other, the server uses the same port numbers the other way<br>around. This might seem very similar to how we would allocate TX and RX lines<br>or pairs on a serial cable, and it is - the NCP protocol design seems a little<br>bizarre to us, given what we know today, but I suspect that at the time it was<br>in fact the obvious and elegant approach; it was structured around the metaphor<br>of existing point-to-point cables.
When NCP gave way to TCP, the same port numbers were retained (TCP was more or<br>less a direct enhancement of NCP). But since TCP was duplex the even-numbered<br>'reverse connection' ports were no longer required and were dropped. Under NCP,<br>FTP used port 21 for one direction and 22 for the other. When TCP replaced NCP,<br>port 21 was retained for FTP but port 22 was no longer needed. Years later, the<br>now-free port 22 was allocated to the up-and-coming SSH protocol. The morals of<br>this story are that, first, if a protocol uses a two-digit odd-numbered port,<br>it is older than I am[1], and second, networking largely happens by accident<br>rather than by design.
As you no doubt suspect, I will be further exploring the latter.
When I said that FTP was historically allocated ports 21 and 22, I lied. In<br>actuality, FTP was historically allocated ports 20, 21, and 22. So let's think<br>this out from principles. NCP even numbered ports were usually referred to as<br>"receive" ports, although I prefer to say server-to-client. So FTP involves not<br>one, but two server-to-client ports, and one client-to-server.
If you have ever issued the PASV command that probably clicks for you, and not<br>in a good way.
Ports 21 and 22 historically, and port 21 today, are better referred to as the<br>FTP control ports. Port 20 is the FTP data port. In the design of FTP,<br>control of the session and transmission of actual data are strictly separate<br>matters handled as separate line protocols on separate connections. But, in<br>order to minimize client-side business logic (keep in mind that this was<br>developed in the mainframe days where the client was potentially a very thin<br>device, 'thin' in the sense that you would call a person 'thick'), data<br>connection initiation was made a server responsibility.
Roughly speaking, here's how it works. Your client connects to the server and<br>starts issuing commands. The server responds to those commands over the same<br>connection (assuming we're in the TCP era where we can talk both ways on one<br>connection). But, when it comes to actually transferring data, which in the<br>case of FTP includes directory listings, the server connects back to the<br>client using the data connection port starts sending data[2].
If you turn your head to the right angle and squint, this is an elegant design.<br>There is a bidirectional control channel used to exchange commands, and then<br>any actual payload is conveyed over a new connection created just for that<br>purpose. The directionality of that convention (or, more importantly to TCP,<br>who initiates it) is dependent on which way the payload is moving, not some<br>more artificial requirement of the protocol design.
Of course, all of this crashes and burns the moment a real modern network gets<br>involved. Today, a large number of client computers are behind NAT, a firewall,<br>or both (and from the outside they don't look that different anyway). When the<br>server tries to connect back to the client to send data, it encounters a<br>rejection or more often complete silence in reply. On the modern internet, the<br>concept of the server initiating a connection to the client is fundamentally<br>untenable. Sometimes by intention and sometimes by mistake, most clients do not<br>have the ability to listen on the internet even if they wanted to.
The FTP protocol had to be modified to account for this...