Level: Beginner to Intermediate\
Role: DevOps Engineer / Cloud Engineer / Site Reliability Engineer\
Project: Restaurant Company\
Environment: Kubernetes (Amazon EKS, kind, or minikube)\
Estimated time: 3–4 hours\
Assignment type: Hands-on Lab + Simulated Production Incident + Interview Preparation

PART 1 — YOUR ROLE AS A DEVOPS ENGINEER

Welcome to Restaurant Company

Imagine you have recently joined Restaurant Company as a DevOps Engineer.

Restaurant Company has a web application that customers use to view restaurant information and place orders.

The application runs in Kubernetes.

Your responsibility is to help ensure that the application is:

• Available to customers.
• Running on healthy Kubernetes Pods.
• Accessible through the correct network routes.
• Deployed reliably through CI/CD pipelines.
• Monitored for failures.
• Recoverable when incidents occur.

You are also responsible for communicating with developers, other DevOps engineers, SREs, engineering managers, and incident responders.

Your job is not simply to execute commands. Your job is to investigate problems, understand the impact, restore service safely, and communicate clearly.

1.1 Tools used by your company

For this simulated environment, assume your company uses:

Different companies may use different tools. The concepts remain similar.

PART 2 — UNDERSTAND THE APPLICATION ARCHITECTURE

Before troubleshooting production issues, you must understand how the application reaches Kubernetes.

2.1 How an application is deployed

A developer writes application code and pushes it to GitHub.

A typical CI/CD workflow then performs the following steps:

1. Developer pushes code to a GitHub repository.
2. GitHub Actions starts a CI/CD pipeline.
3. The pipeline checks out the source code.
4. The pipeline installs dependencies and runs tests.
5. Docker builds a container image.
6. The image is pushed to Amazon ECR.
7. The deployment workflow updates the Kubernetes Deployment to use the new image.
8. The Kubernetes API Server accepts the updated desired state.
9. The Deployment Controller manages the ReplicaSet.
10. The ReplicaSet Controller maintains the required number of Pods.
11. The Scheduler selects suitable Worker Nodes for new Pods.
12. kubelet on each assigned Worker Node works with the container runtime to start the containers.
13. The CNI implementation configures Pod networking.
14. Readiness probes determine whether the application Pods are ready to receive traffic.
15. Kubernetes Service endpoints are updated to reflect eligible backends.

Important: This is the typical production deployment architecture. In this lab, we will create Kubernetes resources manually using YAML and kubectl. We are not building a complete GitHub Actions/ECR pipeline today.

2.2 Kubernetes component responsibilities

API Server

The API Server is the entry point for Kubernetes API requests. When you execute kubectl apply, kubectl sends the requested configuration to the API Server.

etcd

etcd stores Kubernetes cluster state, including the desired state of Kubernetes resources.

Deployment Controller

The Deployment Controller manages ReplicaSets and application rollouts.

ReplicaSet Controller

The ReplicaSet Controller ensures that the desired number of Pods exists.

Scheduler

The Scheduler selects an appropriate Worker Node for a Pod that has not yet been assigned to a Node.

kubelet

kubelet is the agent running on a Kubernetes Node. It works with the container runtime to maintain assigned Pods.

Container Runtime

The container runtime, such as containerd, manages container image pulling and container execution.

Linux Kernel

The Linux Kernel provides the underlying operating system functionality, including processes, networking, namespaces, cgroups, and resource management.

CNI

The Container Network Interface implementation configures Pod networking. Depending on the CNI plugin, this includes interfaces, IP allocation, and routes.

CoreDNS

CoreDNS resolves Kubernetes Service DNS names into their corresponding Service IP addresses.

EndpointSlices

EndpointSlices publish backend endpoint information associated with Kubernetes Services.

kube-proxy

In clusters using kube-proxy, it watches Services and EndpointSlices and configures the Service networking dataplane. In iptables mode, Linux Netfilter processes packets according to the programmed rules.

Other Kubernetes clusters may use different networking implementations, such as eBPF-based Service routing.

PART 3 — YOUR FIRST PRODUCTION TICKET

Incident Ticket: INC-2026-1042

Title: Restaurant Backend Service Unavailable\
Priority: P2 — High (simulated)\
Environment: Kubernetes\
Assigned Team: DevOps / Platform Engineering\
Status: Open — Investigating

Incident description

Customers are reporting that the Restaurant application cannot retrieve backend information.

Monitoring indicates that application requests are failing.

The Kubernetes Deployment appears healthy, but requests to the backend Service are unsuccessful.

Business impact

Customers may be unable to view restaurant information or complete certain application workflows.

Your task is to investigate the issue, identify the root cause, restore connectivity, and document the resolution.

3.1 Where will you receive the ticket?

In a real company, incidents may be reported through:

• Jira or ServiceNow.
• PagerDuty or another on-call alerting system.
• Grafana or Prometheus alerts.
• CloudWatch alarms.
• Slack or Microsoft Teams.
• Customer support escalation.
• Developer reports.

For example:

A monitoring system detects failed HTTP requests and creates an alert.

The alert reaches the on-call engineer.

The engineer acknowledges the alert, reviews the impact, and opens or updates the incident ticket.

The incident is then investigated according to the company's incident-management procedures.

3.2 What should you do first?

Before executing troubleshooting commands:

1. Acknowledge the ticket.
2. Identify the affected application and environment.
3. Determine the reported symptoms and business impact.
4. Check whether there is an active incident bridge or incident commander.
5. Review recent deployments and infrastructure changes.
6. Start with safe, read-only diagnostic commands.
7. Communicate your investigation status.

Do not immediately restart applications, delete Pods, or modify production configuration without understanding the situation and following company procedures.

3.3 Your first Slack message

Example:

Hi team,

I have acknowledged INC-2026-1042 regarding failed requests to the Restaurant backend service.

I am reviewing the Kubernetes Deployment, Pod readiness, Service configuration, and backend endpoints to determine the cause.

I will provide another update within 10 minutes.

Please let me know if there were any recent application deployments or infrastructure changes.

Thank you.

3.4 What information should you collect?

Before troubleshooting, identify:

• Application name.
• Namespace.
• Cluster.
• Environment.
• Time the issue started.
• User-facing symptoms.
• Error codes, if available.
• Recent deployments.
• Monitoring alerts.
• Relevant logs.
• Current incident owner.

For our lab:

Application: Restaurant Company\
Namespace: jumptotech-lab\
Deployment: restaurant\
Service: restaurant-service\
Port: 80\
Replicas: 2

PART 4 — BUILD YOUR KUBERNETES ENVIRONMENT FROM ZERO

Step 1 — Verify Kubernetes access

Open your terminal.

If you are using Amazon EKS, make sure your AWS credentials, cluster access, and kubeconfig are configured.

If you are using kind or minikube, ensure your local cluster is running.

Execute:

kubectl cluster-info
kubectl get nodes -o wide
kubectl config current-context

Expected result:

• Kubernetes API Server is reachable.
• At least one Node is Ready.
• Your current context points to the intended lab cluster.

Important: Never assume you are connected to the correct cluster. Always verify your Kubernetes context before making changes.

If you receive a connection error, stop and troubleshoot cluster access before continuing.

Step 2 — Create your project directory

mkdir -p ~/jumptotech-k8s-network-lab
cd ~/jumptotech-k8s-network-lab
pwd

Step 3 — Create a Namespace

kubectl create namespace jumptotech-lab

Verify:

kubectl get namespaces

Interview question: Why do we use Kubernetes Namespaces?

Answer:

Namespaces provide logical separation of namespaced Kubernetes resources within a cluster. Teams commonly use them to organize applications and environments, apply access controls, and manage resource quotas.

Namespaces alone are not a complete security boundary.

PART 5 — CREATE THE RESTAURANT DEPLOYMENT

Step 4 — Create deployment.yaml

Execute:

cat > deployment.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
  name: restaurant
  namespace: jumptotech-lab
spec:
  replicas: 2
  selector:
    matchLabels:
      app: restaurant
  template:
    metadata:
      labels:
        app: restaurant
    spec:
      containers:
      - name: nginx
        image: nginx:1.27
        ports:
        - containerPort: 80
        resources:
          requests:
            cpu: "50m"
            memory: "64Mi"
          limits:
            cpu: "250m"
            memory: "128Mi"
        readinessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 2
          periodSeconds: 5
EOF

Step 5 — Apply the Deployment

kubectl apply -f deployment.yaml

Check rollout status:

kubectl rollout status deployment/restaurant \
  -n jumptotech-lab --timeout=180s

Check resources:

kubectl get deployments -n jumptotech-lab

kubectl get replicasets -n jumptotech-lab

kubectl get pods -n jumptotech-lab -o wide

Expected result:

• Deployment exists.
• ReplicaSet exists.
• Two Restaurant Pods are Running and Ready.

If the Pods are not Ready, do not continue blindly.

Investigate:

kubectl describe deployment restaurant \
  -n jumptotech-lab

kubectl describe pods \
  -n jumptotech-lab -l app=restaurant

kubectl get events -n jumptotech-lab \
  --sort-by=.lastTimestamp

Step 6 — Understand what happened

When you executed kubectl apply -f deployment.yaml, you submitted the desired application configuration to Kubernetes.

The API Server processed the request.

The Deployment Controller managed the ReplicaSet.

The ReplicaSet Controller created the required Pods.

The Scheduler selected Worker Nodes.

kubelet worked with the container runtime to start the containers.

The CNI implementation configured networking for the Pods.

Student questions:

1. What is a Deployment?
2. What is a ReplicaSet?
3. What is a Pod?
4. Who selects the Worker Node?
5. Who starts the container?
6. What is the role of the Linux Kernel?
7. Why does the Pod need a network interface and IP address?

Be prepared to explain every answer without reading your notes.

PART 6 — CREATE THE KUBERNETES SERVICE

Step 7 — Create service.yaml

cat > service.yaml <<'EOF'
apiVersion: v1
kind: Service
metadata:
  name: restaurant-service
  namespace: jumptotech-lab
spec:
  type: ClusterIP
  selector:
    app: restaurant
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 80
EOF

Apply:

kubectl apply -f service.yaml

Verify:

kubectl get svc -n jumptotech-lab

kubectl describe svc restaurant-service \
  -n jumptotech-lab

Check EndpointSlices:

kubectl get endpointslices \
  -n jumptotech-lab \
  -l kubernetes.io/service-name=restaurant-service \
  -o wide

Step 8 — Understand Service and Pod labels

Your Service has this selector:

selector:
  app: restaurant

Your Pods have this label:

labels:
  app: restaurant

The matching labels allow the EndpointSlice Controller to publish the correct backend endpoints.

The Service provides a stable virtual IP address, called a ClusterIP.

Pods can be replaced and receive new IP addresses, while the Service can retain its ClusterIP.

Interview question: Why do we need a Service if Pods already have IP addresses?

Answer:

Pod IP addresses may change when Pods are recreated. A Kubernetes Service provides a stable endpoint and Service discovery mechanism for reaching eligible backend Pods.

PART 7 — CREATE A DIAGNOSTIC CLIENT POD

Step 9 — Create client.yaml

cat > client.yaml <<'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: client
  namespace: jumptotech-lab
spec:
  containers:
  - name: netshoot
    image: nicolaka/netshoot:latest
    command: ["sleep", "36000"]
    resources:
      requests:
        cpu: "25m"
        memory: "32Mi"
      limits:
        cpu: "200m"
        memory: "128Mi"
EOF

Apply:

kubectl apply -f client.yaml

Wait:

kubectl wait \
  --for=condition=Ready \
  pod/client \
  -n jumptotech-lab \
  --timeout=180s

Enter the Pod:

kubectl exec -it client \
  -n jumptotech-lab -- bash

Your terminal prompt should change to something similar to:

client:~#

Important: You are now inside the Client Pod, not on your EC2 management host.

PART 8 — INVESTIGATE POD NETWORKING

Step 10 — Check the Pod hostname

Inside Client Pod:

hostname

Step 11 — Check network interfaces

ip addr

Find the eth0 interface and its IPv4 address.

Record the IP address.

Step 12 — Check routing

ip route

Look for the default route.

In AWS VPC CNI, you may see a gateway such as:

default via 169.254.1.1 dev eth0

Other Kubernetes network implementations may have different routing configurations.

Step 13 — Check DNS configuration

cat /etc/resolv.conf

Identify the configured DNS server and search domains.

Interview question: What is a Linux Network Namespace?

Answer:

A Network Namespace provides an isolated view of networking resources, including network interfaces, IP addresses, routing tables, and ports.

Containers in the same Kubernetes Pod normally share one Network Namespace.

Interview question: What does CNI do?

Answer:

The CNI implementation configures networking for Pods. Depending on the plugin, this includes network interfaces, IP addresses, and routing.

PART 9 — TRACE AN HTTP REQUEST

Step 14 — DNS resolution

Still inside Client Pod:

nslookup restaurant-service

Then:

nslookup restaurant-service.jumptotech-lab.svc.cluster.local

Expected:

The DNS lookup returns the ClusterIP assigned to restaurant-service.

Record the IP address.

What happened?

The Client Pod's resolver queried cluster DNS.

CoreDNS resolved the Kubernetes Service name.

The DNS response provided the Service IP address.

CoreDNS does not forward the HTTP request itself.

Step 15 — Send HTTP request

curl -v http://restaurant-service

Then:

curl -I http://restaurant-service

Expected HTTP status:

HTTP/1.1 200 OK

The default NGINX welcome page should be returned.

Step 16 — Understand the complete internal request flow

1. Client Pod initiates an HTTP request.
2. DNS resolves restaurant-service to its ClusterIP.
3. The client establishes a TCP connection to the Service IP on port 80.
4. The Kubernetes Service dataplane selects an eligible backend Pod.
5. In iptables-based kube-proxy mode, Linux Netfilter applies Service forwarding and address translation rules.
6. The packet reaches the selected Restaurant Pod.
7. NGINX receives the HTTP request on port 80.
8. NGINX returns the HTTP response.
9. The response travels back to the Client Pod.

Important distinction:

kube-proxy programs forwarding rules in its supported modes. In iptables mode, the Linux Kernel processes packets using those rules.

The exact implementation depends on the cluster networking configuration.

Step 17 — Compare direct Pod IP requests

Exit the Client Pod:

exit

Find the current Restaurant Pod IP addresses:

kubectl get pods -n jumptotech-lab \
  -l app=restaurant -o wide

Record both Pod IPs.

Enter Client Pod again:

kubectl exec -it client \
  -n jumptotech-lab -- bash

Run the following, replacing the placeholders with the real IP addresses:

curl -I http://POD_1_IP:80
curl -I http://POD_2_IP:80

For example, if a Pod IP is 172.31.46.36:

curl -I http://172.31.46.36:80

Interview question: What is the difference between a Service request and a direct Pod IP request?

Answer:

A Service request uses a stable Service endpoint and the cluster's Service routing implementation to reach an eligible backend.

A direct Pod IP request targets one specific Pod and bypasses Service load balancing.

PART 10 — REPRODUCE THE PRODUCTION INCIDENT

Now we will simulate a real Kubernetes networking incident.

We will intentionally create a Service with an incorrect selector.

We will not modify the healthy Restaurant Service.

Step 18 — Exit Client Pod

exit

Step 19 — Create a broken Service

cat > broken-service.yaml <<'EOF'
apiVersion: v1
kind: Service
metadata:
  name: restaurant-broken
  namespace: jumptotech-lab
spec:
  type: ClusterIP
  selector:
    app: wrong-restaurant
  ports:
  - name: http
    port: 80
    targetPort: 80
EOF

Apply:

kubectl apply -f broken-service.yaml

Step 20 — Investigate the broken Service

Check:

kubectl get svc -n jumptotech-lab

Describe:

kubectl describe svc restaurant-broken \
  -n jumptotech-lab

Check Pod labels:

kubectl get pods -n jumptotech-lab \
  --show-labels

Check EndpointSlices:

kubectl get endpointslices \
  -n jumptotech-lab \
  -l kubernetes.io/service-name=restaurant-broken \
  -o wide

Observation:

The Service selector uses app: wrong-restaurant.

The Restaurant Pods use app: restaurant.

Because these values do not match, the Service has no eligible backend endpoints.

Step 21 — Reproduce the failed HTTP request

Test DNS from Client Pod:

kubectl exec client -n jumptotech-lab -- \
  nslookup restaurant-broken

The DNS name should resolve to the broken Service's ClusterIP.

Now test HTTP:

kubectl exec client -n jumptotech-lab -- \
  curl -v --max-time 5 http://restaurant-broken

The request should fail because the Service has no eligible backend endpoints.

The exact failure may be connection refused, timeout, or another network-level error depending on the cluster dataplane.

Do not automatically describe this as HTTP 503. A Service without endpoints does not necessarily return an HTTP response.

Step 22 — Identify the root cause

Compare:

Service selector:

app: wrong-restaurant

Pod label:

app: restaurant

Root cause:

The Kubernetes Service selector did not match the Restaurant Pod labels.

As a result, no eligible backend endpoints were available for that Service.

The application Pods were healthy, but traffic could not reach them through the broken Service.

PART 11 — COMMUNICATE YOUR FINDINGS

Before applying the fix, write an investigation update.

Example Slack message:

Hi team,

I identified a likely root cause for INC-2026-1042.

The Restaurant application Pods are Running and Ready, but the affected Kubernetes Service has no eligible backend endpoints.

The Service selector is configured with app: wrong-restaurant, while the application Pods use app: restaurant.

I reproduced the connectivity failure from a diagnostic Pod.

I am preparing a selector correction and will verify the backend endpoints and HTTP response after the change.

PART 12 — FIX THE INCIDENT

Step 23 — Correct the Service selector

For our isolated lab, execute:

kubectl patch svc restaurant-broken \
  -n jumptotech-lab \
  -p '{"spec":{"selector":{"app":"restaurant"}}}'

In real production, follow change authorization and GitOps procedures. If the Service is managed through Git, Helm, or Argo CD, update the source of truth to prevent the incorrect configuration from returning.

Step 24 — Verify the fix

kubectl describe svc restaurant-broken \
  -n jumptotech-lab

Check EndpointSlices:

kubectl get endpointslices \
  -n jumptotech-lab \
  -l kubernetes.io/service-name=restaurant-broken \
  -o wide

Confirm that backend Pod IPs appear.

Then:

kubectl exec client -n jumptotech-lab -- \
  curl -I --max-time 5 http://restaurant-broken

Expected:

HTTP/1.1 200 OK

Step 25 — Verify the original healthy Service

kubectl exec client -n jumptotech-lab -- \
  curl -I --max-time 5 http://restaurant-service

Expected:

HTTP/1.1 200 OK

For a real production incident, one successful curl is not enough to establish full recovery.

You should also verify monitoring metrics, error rates, readiness, backend health, and the actual user-facing path.

PART 13 — RESOLUTION COMMUNICATION

After verifying recovery, write a resolution update.

Example Slack message:

Hi team,

INC-2026-1042 has been resolved in our lab environment.

Root cause: The Kubernetes Service selector did not match the application Pod labels, leaving the Service without eligible backend endpoints.

Actions taken:

• Confirmed the application Pods were Running and Ready.
• Reviewed Service configuration and Pod labels.
• Inspected EndpointSlices.
• Reproduced the failed HTTP request from a diagnostic Pod.
• Corrected the Service selector.
• Verified backend endpoints were populated.
• Confirmed HTTP 200 responses after the fix.

I recommend adding automated validation for Service selectors and backend endpoint availability to help prevent similar configuration errors.

I will document the investigation and resolution in the incident ticket.

Important: Only use this resolution message after completing and verifying the corresponding actions.

PART 14 — COMPLETE YOUR INCIDENT TICKET

Incident Report

Incident ID: INC-2026-1042

Application: Restaurant Company

Environment: Kubernetes Lab

Issue: Backend Service connectivity failure.

Impact: Simulated inability to reach the backend application through the affected Service.

Detection: Simulated monitoring alert and failed HTTP connectivity test.

Investigation:

Reviewed Kubernetes Deployment, Pod status, Service configuration, Pod labels, and EndpointSlices.

Confirmed the application Pods were Running and Ready.

Observed that the affected Service did not have eligible backend endpoints.

Root Cause:

Service selector mismatch.

The Service used app: wrong-restaurant, while the Pods used app: restaurant.

Resolution:

Updated the Service selector to match the Pod labels.

Verification:

Confirmed backend endpoints appeared and the Service returned HTTP 200 from the diagnostic Client Pod.

Preventive Actions:

• Add deployment validation for Service selectors.
• Monitor Service backend availability.
• Review Kubernetes manifests before deployment.
• Use GitOps and peer-reviewed changes where applicable.
• Maintain clear incident runbooks.

Status: Resolved — after successful verification.

PART 15 — BONUS LAB: KUBERNETES SELF-HEALING

This lab demonstrates how Kubernetes maintains the desired number of Pods.

Step 26 — Check existing Pods

kubectl get pods -n jumptotech-lab \
  -l app=restaurant

Step 27 — Delete one Pod

Choose an actual Restaurant Pod name.

kubectl delete pod POD_NAME \
  -n jumptotech-lab

Replace POD_NAME with the real name.

Step 28 — Watch Kubernetes create a replacement

kubectl get pods -n jumptotech-lab -w

Press Ctrl+C when the new Pod becomes Ready.

Step 29 — Verify replicas

kubectl get deployment restaurant \
  -n jumptotech-lab

Expected: Two Ready replicas after recovery.

Interview question: What happens when a Pod is deleted?

Answer:

The ReplicaSet Controller detects that the number of existing Pods is below the desired replica count and creates a replacement Pod.

The Scheduler assigns the new Pod to a suitable Node, and kubelet starts its containers.

This is an example of Kubernetes reconciliation and self-healing.

PART 16 — BONUS LAB: ACCESS THE APPLICATION FROM A BROWSER

From the machine where kubectl is configured:

kubectl port-forward \
  -n jumptotech-lab \
  svc/restaurant-service \
  8080:80

Open:

http://localhost:8080

Expected: NGINX welcome page.

If kubectl is running on a remote EC2 instance, localhost:8080 refers to that EC2 instance, not your laptop. Use an approved SSH tunnel or another authorized access method if you need to view it locally.

Important: Port-forwarding is a diagnostic tool. It does not demonstrate the production AWS ALB and Ingress request path.

PART 17 — UNDERSTAND THE EXTERNAL HTTP REQUEST FLOW

A real customer usually does not access the application's internal Kubernetes Service name.

Consider a production architecture using AWS ALB.

A typical request flow is:

1. Customer opens the Restaurant Company website.
2. The browser resolves the public domain through DNS.
3. The browser connects to the public application endpoint, commonly over HTTPS.
4. AWS ALB receives the request.
5. ALB evaluates listener and routing rules.
6. The request is forwarded to a configured target group.
7. The target group sends the request to a healthy backend target.
8. The application processes the request.
9. The HTTP response returns to the browser.

With AWS Load Balancer Controller, ALB can target Pod IPs directly in IP target mode or NodePorts in instance target mode.

For a separate internal backend call, a frontend server Pod may resolve a Kubernetes Service name using CoreDNS and send traffic through the internal Service dataplane.

A browser-hosted React application may instead call a public backend API through ALB.

You must know which architecture your application actually uses.

Important HTTP status codes

Important: A 404 does not necessarily mean the entire application is unavailable. A 400 does not automatically mean the application is down. HTTP status codes must be interpreted in context.

PART 18 — HOW TO TROUBLESHOOT DIFFERENT ERRORS

Scenario A — Pod is Pending

Check:

kubectl describe pod POD_NAME -n jumptotech-lab

Investigate:

• Insufficient CPU or memory.
• Scheduling constraints.
• Node availability.
• Taints and tolerations.
• Persistent volume requirements.

Scenario B — ImagePullBackOff

Check:

kubectl describe pod POD_NAME -n jumptotech-lab

Investigate:

• Incorrect image name.
• Missing image tag.
• Registry access.
• Image pull permissions.
• Registry connectivity.

Scenario C — CrashLoopBackOff

Check:

kubectl logs POD_NAME \
  -n jumptotech-lab --previous

Also inspect Pod events and container exit codes.

Investigate:

• Application startup failure.
• Incorrect configuration.
• Missing dependencies.
• Resource limits.
• Probe failures.

Scenario D — Service DNS resolution fails

Check from a diagnostic Pod:

nslookup restaurant-service

Investigate:

• Service existence.
• Namespace and DNS name.
• CoreDNS health.
• Pod DNS configuration.
• DNS connectivity and network policies.

Scenario E — DNS works but HTTP fails

Check:

kubectl get svc -n jumptotech-lab

kubectl get endpointslices \
  -n jumptotech-lab

kubectl get pods -n jumptotech-lab \
  --show-labels

Investigate:

• Service selector.
• Pod readiness.
• Target port.
• Application listener.
• Network policies.
• Service dataplane behavior.

Scenario F — External users receive 503

Investigate:

• AWS ALB target health.
• Ingress configuration.
• Target groups.
• Service and endpoint availability.
• Pod readiness.
• Application health checks.

The investigation depends on which component generated the 503.

PART 19 — HOW TO ANSWER IN A DEVOPS INTERVIEW

Interview Question 1: Tell me about yourself

Sample answer:

I am building hands-on experience in DevOps and cloud infrastructure, focusing on Linux, AWS, Docker, Kubernetes, Terraform, and CI/CD automation.

Through my Restaurant Company project, I have practiced deploying containerized applications to Kubernetes, configuring Deployments and Services, troubleshooting Pod networking, and investigating application connectivity issues.

I have also worked on simulated production incidents, where I followed a structured process to identify problems, verify fixes, and communicate technical findings.

I am especially interested in cloud infrastructure automation, Kubernetes operations, and production reliability.

Interview Question 2: Explain how an application gets deployed to Kubernetes

Sample answer:

In a typical CI/CD workflow, developers push their code to GitHub, which triggers an automated pipeline.

The pipeline builds and tests the application, creates a Docker image, and pushes it to a container registry such as Amazon ECR.

The deployment process updates the Kubernetes Deployment with the new image version.

The Kubernetes API Server accepts the desired configuration, and the Deployment Controller manages the rollout through ReplicaSets.

The Scheduler assigns new Pods to suitable Worker Nodes, and kubelet works with the container runtime to start the containers.

The CNI implementation configures Pod networking, and readiness probes help determine when the application is ready to receive traffic.

Interview Question 3: Explain how an HTTP request reaches a Kubernetes application

Sample answer:

For an external request, the client first resolves the public domain through DNS and connects to the application's public endpoint.

In an AWS environment, the request may reach an Application Load Balancer.

The ALB evaluates its listener rules and forwards traffic to a healthy backend target.

For internal communication between Kubernetes applications, a client Pod can use the backend Service DNS name.

CoreDNS resolves the Service name to its ClusterIP.

The Kubernetes Service dataplane then forwards the connection to an eligible backend Pod.

The application processes the HTTP request and sends the response back to the client.

Interview Question 4: Tell me about a Kubernetes incident you resolved

Sample answer:

During my DevOps training, I investigated a simulated production incident involving a Restaurant Company application running in Kubernetes.

The application Pods were healthy, but HTTP requests through a Kubernetes Service were failing.

I started by reviewing Pod readiness, the Service configuration, Pod labels, and EndpointSlices.

I reproduced the connectivity failure from a diagnostic Client Pod.

DNS resolution was successful, which helped me separate DNS issues from Service routing problems.

I found that the Service selector did not match the application Pod labels, leaving the Service without eligible backend endpoints.

I corrected the selector, verified that the expected endpoints appeared, and confirmed the application returned HTTP 200.

I also practiced communicating the incident status, documenting the root cause, and recommending preventive checks.

This exercise helped me understand how Kubernetes networking works and how to approach connectivity issues systematically.

Interview Question 5: How do you troubleshoot a Kubernetes application that is unavailable?

Sample answer:

I first confirm the scope and impact of the issue.

I check monitoring alerts and determine whether the problem affects all users or only specific requests.

Then I inspect the Kubernetes Deployment, Pod readiness, events, and application logs.

If the Pods are healthy, I investigate Service configuration, EndpointSlices, DNS resolution, and network connectivity.

For externally exposed applications, I also check Ingress configuration, load balancer target health, and relevant AWS networking components.

Once I identify the likely root cause, I coordinate the appropriate mitigation, verify recovery, monitor for recurrence, and document the incident.

Interview Question 6: What is the difference between CoreDNS, CNI, and kube-proxy?

Sample answer:

CoreDNS provides DNS resolution for Kubernetes Service names.

CNI is the interface through which the networking implementation configures Pod connectivity, including IP addressing and interfaces.

kube-proxy, when used, implements Kubernetes Service networking by programming forwarding rules based on Service and EndpointSlice information.

These components have different responsibilities, but they work together to support communication between Kubernetes applications.

Interview Question 7: How do you communicate during a production incident?

Sample answer:

I acknowledge the incident and confirm the business impact.

I provide regular updates to the incident channel, including what has been verified, what I am investigating, and when the next update is expected.

I avoid presenting assumptions as confirmed root causes.

If the incident requires escalation or a potentially disruptive change, I coordinate with the incident lead and relevant teams.

After service recovery, I document the root cause, resolution, verification results, and preventive actions.

PART 20 — YOUR FINAL HOMEWORK

Submit the following:

1. Screenshot showing Kubernetes Nodes in Ready status.
2. Screenshot showing your Restaurant Deployment and two Ready Pods.
3. Screenshot showing your Kubernetes Service and ClusterIP.
4. Screenshot showing the EndpointSlices.
5. Screenshot showing successful CoreDNS resolution.
6. Screenshot showing HTTP 200 from the healthy Service.
7. Screenshot showing the broken Service selector.
8. Screenshot showing missing backend endpoints.
9. Screenshot showing the failed HTTP request.
10. Screenshot showing the corrected selector.
11. Screenshot showing HTTP 200 after the fix.
12. Your first Slack incident acknowledgment.
13. Your investigation update.
14. Your resolution update.
15. Your completed incident report.
16. A written or recorded interview explanation of the incident.

Final practical assessment

Your instructor may ask you to share your screen and demonstrate:

• How to find a Deployment.
• How to check Pod health.
• How to find the Pod IP.
• How to inspect a Service.
• How to inspect EndpointSlices.
• How to check DNS resolution.
• How to test HTTP connectivity.
• How to identify a selector mismatch.
• How to correct the configuration.
• How to verify recovery.

You should explain what every command does and why you are using it.

PART 21 — CLEANUP

After your instructor confirms that the assignment is complete, remove the lab resources:

kubectl delete namespace jumptotech-lab

This removes the resources created in that Namespace.

Do not run this command until you have saved your screenshots and completed the assignment.

FINAL MESSAGE TO STUDENTS

Your goal is not to memorize Kubernetes commands. Your goal is to understand the full lifecycle of an application and how to troubleshoot it when something goes wrong.

A strong DevOps Engineer should be able to explain:

1. How application code moves from GitHub to Kubernetes.
2. How Kubernetes schedules and runs containers.
3. How Pods receive network connectivity.
4. How Services and EndpointSlices support application communication.
5. How DNS resolution differs from HTTP routing.
6. How an HTTP request reaches an application.
7. How to investigate a production incident safely.
8. How to communicate findings and progress.
9. How to verify that a problem is resolved.
10. How to explain the work clearly in a technical interview.

Remember the incident response structure:

Impact → Investigation → Evidence → Root Cause → Mitigation → Verification → Communication → Prevention

Practice explaining your project in your own words.

Be clear about which activities you performed in a lab and which tools or processes belong to the example production architecture. Never present a training exercise as actual employment experience.

JumpToTech DevOps Bootcamp — Learn It. Build It. Troubleshoot It. Explain It.

You can copy this entire handbook directly into your student group or development platform. It includes the complete lab, the simulated ticket, step-by-step commands, workplace communication, incident reporting, and interview preparation.