If you have had an incident in your production environment, you might have an IP address of the offending pod, but not knowing which container it is.
This is because by the time you have identified the IP of the offending pod, by that time the pod has already been cycled, killed and removed from your Kubernetes cluster.
A good example of this is when you have a slow query in your database, and your database is reporting the source IP of the query, but not which pod.
View API-Server Audit Logs
The Kubernets event log keeps a history of IP assignments, but an easier place to search would be the API-Server logs in the GKE audit logs, which is under the Logs tab for your GKE cluster.
Search for your IP
With your IP in-hand you can actually just simply search the GKE audit logs for any matches.
You will be able to see the exact moment the IP as assigned, but also unassigned.
You will be able to also look back historically through the logs to ensure that the pod that caused the issue was exactly the issue is the exact one you think it is.
In this example at 2024-09-20T15:51:07.027242Z
the IP address was unassigned from system:install-argocd-repo-server
.
Then at 2024-09-20T18:04:27.582350Z
I can see that it was assigned to ci:ci-registry
.