Open a shell and browse files in an instance
Open a terminal in a running instance from the browser, and list, download and upload its files. Covers who may do it, protected environments, the debug container fallback, limits and what is recorded.
When something misbehaves inside a running service, you often need to look: run a command, read a config file, copy a log out. This guide opens a shell in one instance from the console and browses its files, with no kubeconfig.
Before you begin
- You have the admin or owner role. A shell runs as the container's user with the container's environment, so it can read every variable the service holds, secrets included. Deployers and viewers do not see the buttons.
- The service has a running instance. A database has no shell; use the Data tab to query it.
- The cluster's agent has the permission to open a shell. An agent update changes only its image. A cluster installed before the shell existed needs its install manifest applied again for the agent's role to include
pods/exec. If the console says Update the agent to open a shell, follow Re-apply the manifest on an older cluster. - For a protected environment such as production, an administrator has turned on Allow shell in protected environments (see below).
Open a shell
Open the service, then its Instances tab.
On the instance's row, select Open shell. A terminal opens under the list and connects to the instance's main container.
Type commands as you would in any terminal. Copy and Paste sit above it; Ctrl+Shift+C and Cmd+C copy the selection, and Ctrl+C with nothing selected interrupts the running command. Resizing the panel resizes the terminal.
Select Close when you are done, or type exit.
If the connection drops, the terminal says so and offers Open a new shell. A dropped connection ends the session; it is not resumed, and nothing in the pod changes.
When the container has no shell
If the container has no /bin/sh (a distroless image, for example), Nebula adds a short-lived debug container to the pod instead. It runs a pinned busybox image, shares the container's processes and network, and runs as the same user with every capability dropped. The terminal tells you when this happens.
In the debug container the target's files are under /proc/1/root, and the shell starts there. The first time on a node the image is pulled, which can take a moment; if the node cannot reach Docker Hub, the terminal says the pull failed and the pod is unchanged.
A debug container stays in the pod until the pod is replaced by a restart or a deploy. Later sessions reuse it while it is running.
Browse and download files
On the instance's row, select Files.
Select a folder to open it, or use the path above the list to go back. Select Download to save a file, or Download .tar to save a folder as an archive.
Files are read by running ls, cat and tar in the container, or in the debug container when it has none of them. A transfer is at most 10 MiB; a larger file is refused and nothing is changed.
Uploading is a separate permission, also held by administrators and owners. Select Upload here to write a file into the folder you are viewing. A file of the same name in that folder is replaced. The file is written as the container's user, so it fails where that user cannot write.
Protected environments
A shell in a protected environment is refused unless the organization allows it. An administrator turns the policy on under Settings > Security & policy > Change policy with the Allow shell in protected environments switch. It is off by default, applies to every protected environment of the organization, and takes effect for sessions already open within a minute.
Limits
| Limit | Value |
|---|---|
| Who | Administrators and owners, and API tokens with the Admin role |
| Session length | 1 hour, then the session ends |
| Idle time | 15 minutes with no input or output, then the session ends |
| Open shells | 5 per person or API token |
| File transfer | 10 MiB per download or upload |
A session also ends when your sign-in ends, your role drops below administrator, or the policy is switched off.
What is recorded
Every session writes two entries to the activity log: when it starts and when it ends, with who opened it, the instance and container, and how long it lasted and why it ended. A session that could not open is recorded as a failure. Each file listing, download and upload is recorded with the path.
The commands you type and their output are not recorded. They pass through the control plane and the agent as a stream and are not stored. If you need a record of what was run in production, do not rely on the shell.
How it connects
Your browser opens a WebSocket to the control plane with a one-time ticket that the console obtained when it authorized you. The ticket works once and expires after 30 seconds, and the socket accepts a connection only from the console's own origin. The control plane asks the cluster's agent, over the connection the agent already holds, to open the session. The agent dials its own WebSocket back to the control plane for that session, authenticated with its cluster token, and attaches it to the pod through the Kubernetes API. An agent can attach only to a session of its own cluster. The control plane relays between the two sockets. Nothing opens a port on your cluster.
The agent can exec only into environment namespaces the access broker has authorized, in pods that are instances of a Nebula service.
Troubleshooting
- Update the agent to open a shell. The cluster's agent predates the shell, or its permissions were not refreshed. Update the agent from the cluster's page, then re-apply the manifest on the cluster so the agent's role includes
pods/exec. An update alone never changes roles. - Shell access is off for protected environments. Turn on Allow shell in protected environments, or use a non-production environment.
- The cluster's agent is not connected. Check the cluster's status. No shell is opened while the agent is away.
- The debug container did not start. The node could not pull the
busyboximage. Check the node's network access to Docker Hub.
Verify
Run hostname in the terminal. It prints the name of the instance you picked. Select Close, then open Activity in the organization's navigation. Two entries appear: "opened a shell in INSTANCE" and "closed the shell in INSTANCE", each with the service.
Next steps
Read logs and metrics
Tail and search a service's logs, read build and deploy logs, and read the Metrics tab, the numbers on each canvas card and the canvas Traffic view. Also covers opting a worker's own metrics in.
Get alerted when a service crosses a threshold
Turn on, tune and acknowledge the alert rules of a service. Covers the four rules (volume usage, memory, restart loops and CPU), when an alert opens and resolves, the canvas pill and Alerts panel, and who is notified.