Dependencies should be fetched directly from VCS
Personal website of Martin Tournoij (“arp242 ”);<br>writing about programming (CV) and various other things.
Working on<br>GoatCounter and<br>more –<br>GitHub Sponsors.
Contact at<br>martin@arp242.net or<br>GitHub.
I’ve been writing Ruby at my new $dayjob in the last month. After spending most<br>of the last decade writing Go it’s been a fun change of scenery. I did Ruby<br>before (years ago) and I can’t really tell you which is “better” – Ruby is very<br>different from Go in almost every respect and I find both quite effective in<br>getting stuff done in their own way.
One aspect where I do feel Go is clearly better is dependency management;<br>specifically the security aspect thereof. Go is not magically immune to<br>malicious dependencies, but it is a lot more resistant to them chiefly because<br>there is no “publish a package” step.
In Go dependencies are identified by URL, e.g. github.com/user/pkg. Go<br>identifies which VCS is being used (git in this case) and fetches the tag or<br>commit you specified in your go.mod file. The go.mod file serves as both a<br>dependency specification and “lock file”. It lists exact versions; there is no<br>~>1.1. It includes both direct and indirect dependencies and lists your full<br>dependency tree (the go command writes to go.mod).
A hash of all files is checked against known hashes on sum.golang.org to prevent<br>tags from being replaced, and it uses a proxy to prevent repos from being<br>left-pad’d.
There are more aspects to Go Modules, including security features, but I will<br>skip over them for the purpose of this article. The relevant bit is<br>“dependencies are identified by URL, the code is fetched directly from the VCS,<br>and it does this for both direct and indirect dependencies”.
Auditing and updating dependencies is easy: I do git log -p old..new (usually<br>via a forge web UI), read all the commits, and update the go.mod file. I don’t<br>have many dependencies and those I do have don’t change much. It’s usually<br>pretty fast. I don’t need to do careful in-depth reviews here; just look for<br>suspicious stuff. Something like exec.Command(..) or http.Post(..) in a<br>globbing library would stand out. It’s hard to really hide stuff.
I’ve been doing this for years for every dependency. As a solo developer. It’s<br>easy. Some projects have much larger dependency trees and this becomes more<br>time-consuming, but not hard or confusing. It’s still easy, just takes a bit of<br>time.
For Ruby things are different as it has a “publish a package” step: you create a<br>.gem archive and upload that to rubygems.org. You can put anything in there – no<br>guarantee the .gem contents correspond to the source repo. To audit it I need to<br>do something like:
curl -s https://rubygems.org/downloads/example-2.7.5.gem >old.gem<br>curl -s https://rubygems.org/downloads/example-2.8.2.gem >new.gem
mkdir old new<br>tar xf old.gem -C old<br>tar xf new.gem -C new
(cd old && tar xf data.tar.gz)<br>(cd new && tar xf data.tar.gz)
diff -urN old new
It works, I guess, but is far from easy. The individual commits are lost and is<br>generally harder to audit. In some cases the diff is small enough that it’s<br>okay. In other cases it’s huge and not having access to the commits is a pain. I<br>can also totally see myself get confused about what I did and didn’t audit. I<br>guess the number of people doing this sort of auditing is very low because it’s<br>just such a pain.
This is not unique to Ruby: this is how many (most?) package systems work.
Almost all of the “side-channel attacks” I’ve seen are perhaps more accurately<br>described as “package publishing attacks”. They rely on injecting something in<br>the “publish a package” step. Whether that’s RubyGems, npm, PyPI, .tar.gz FTP<br>downloads, or something else is a relatively minor detail. It’s rare that the<br>actual source repo gets compromised as it’s just too visible. You need to at<br>least slightly hide your exploit for it to be effective.
The recent npm compromises all relied on gaining access to the npm account and<br>injecting something in the published package. xz had some exploit code in the<br>source repo but was inert, hidden in a binary test file and only activated in<br>the modified .tar.gz release. Back in 2018 event-stream added a dependency on<br>flatmap-stream, which had nefarious code in index.min.js only on the published<br>npm package.
Which points to a second problem: packages containing “compiled” resources. The<br>JavaScript that TypeScript generates is not completely unreadable, it’s<br>certainly much less readable and auditable than the original TypeScript. To<br>say nothing of minified files or binary blobs. This is less of a problem in<br>Ruby, but a far bigger problem in npm.
Last week RubyGems added a cooldown option and “AI-assisted vulnerability<br>scanning against the most critical gems”. Not a bad thing to do as a short-term<br>move, but I feel a more appropriate solution would be to reconsider the entire<br>“publish a package” model. It just lacks the required transparency. AI tools are<br>not going to...