DevOps Engineer Top 100 Interview Questions

Infrastructure as code means defining servers, networks, databases and other infrastructure in version-controlled files, and creating or changing them with automated tools instead of by hand in a console.

• Repeatability: the same files create identical environments, so staging truly matches production.
• Review and history: infrastructure changes go through the same review and approval as application code.
• Speed: a whole environment can be created in minutes, for example a temporary test environment per feature.
• Disaster recovery: if a region or account is lost, infrastructure can be recreated from code.
• Fewer configuration errors: no forgotten manual steps or one-off settings.
• Documentation: the code shows exactly what exists and how it is configured.

The benefits only hold if the code stays the single source of truth. Manual changes in a console create drift, so teams usually restrict console write access and make every change through the code and its pipeline.

Declarative code describes the end state you want, and the tool works out the steps to get there. Imperative code lists the exact steps to run, in order.

When to use which:

• Use declarative definitions for most infrastructure, because they are idempotent and show drift clearly.
• Use imperative scripts for one-off procedures, complex ordering, or tasks the declarative tool cannot express, such as a data migration.

Many tools mix both: declarative resources with small imperative hooks, which should be kept short and idempotent.

Many declarative infrastructure tools keep a state file that maps each resource in the code to the real resource it created, along with its known attributes. The tool compares code, state and reality to work out what to change.

Why state must be central:

• If each engineer keeps a local copy, the copies diverge and the tool may try to recreate or delete resources that already exist.
• A shared remote location gives everyone, including the pipeline, the same view.

Why it must be locked:

• Two people applying changes at the same moment can corrupt the state or leave infrastructure half-changed.
• A lock lets one operation run at a time; the second waits or fails clearly.

How to protect it:

1. Store it in remote storage with versioning, so a damaged state can be restored.
2. Encrypt it and restrict access, because state often contains sensitive values such as generated passwords.
3. Enable locking for every change.
4. Split large estates into several smaller states, so one change does not risk everything.

Interview tip: point out that state may contain secrets in plain text, which is why it needs the same protection as a credential store.

Configuration drift is the gap that grows between the infrastructure defined in code and what is actually running, usually because of manual changes, emergency fixes or automated processes outside the code.

Why it matters:

• The code no longer describes reality, so recreating an environment gives a different result.
• The next automated change may undo a manual fix, or fail unexpectedly.
• Security settings changed by hand can quietly weaken protection.

How to detect it:

1. Run a scheduled comparison of code against real infrastructure and alert on any difference.
2. Audit console and command-line changes made outside the pipeline.
3. Use pull-based agents that continuously reconcile and report drift.

How to fix it:

• If the manual change was wrong, re-apply the code to restore the declared state.
• If the manual change was right, add it to the code first, then apply, so code and reality match.
• Reduce future drift by removing manual write access and making the pipeline the only path for changes.

An operation is idempotent when running it once or many times gives the same result. Idempotent automation can be re-run safely after a failure or on a schedule, without creating duplicates or errors.

• Declarative tools are idempotent by design: if the server already exists with the right settings, applying again changes nothing.
• Scripts must be made idempotent: check the current state before acting, such as creating a directory only if it is missing.
• Retries become safe: a pipeline step that failed halfway can simply run again.
• Drift correction depends on it: a scheduled run fixes only what has changed.

Common ways to write idempotent steps:

1. Check, then act: test whether a line, user or file already exists.
2. Use commands that are idempotent by nature, such as mkdir -p.
3. Write the full desired content of a file instead of appending to it.
4. Use unique keys, so repeating a write updates instead of duplicating.

A non-idempotent script, such as one that appends a line to a configuration file on every run, slowly breaks the system it was meant to manage.

With mutable infrastructure, servers are updated in place over their lifetime. With immutable infrastructure, a server is never changed after it starts: any change means building a new image and replacing the old server.

When to use which:

• Prefer immutable infrastructure for application servers and containers, because every instance is identical and rollback is simple.
• Mutable infrastructure is still common for databases and systems that hold state, where replacement is expensive; manage them with idempotent configuration code.

Immutable infrastructure depends on automation: building images, rolling out replacements and keeping state outside the servers.

Share one set of reusable modules across environments, keep each environment's differences in small configuration files, and give every environment its own state and credentials.

1. Reusable modules: define a network, a database or a service once, with inputs for size, count and names.
2. Thin environment layers: each environment calls the same modules with its own values, such as smaller machines in development.
3. Separate state: development, staging and production each have their own state, so a mistake in one cannot touch another.
4. Separate access: production credentials are available only to the production pipeline.
5. Versioned modules: environments pin module versions, so a module change is tested in development before staging and production adopt it.

Trade-offs:

• Copying code per environment seems simple but drifts quickly; modules keep environments consistent.
• Too many inputs make modules hard to use, so expose only the settings that really differ.

Generate a preview of exactly what will change, review that preview like a code change, and apply exactly the reviewed plan through the pipeline.

1. Open a change request with the code change.
2. Generate a plan automatically: the pipeline shows which resources will be created, changed or destroyed.
3. Run policy checks: for example, no public storage, required tags, and no deletion of protected resources.
4. Human review: check the plan, not just the code, paying close attention to anything destroyed or replaced.
5. Apply the saved plan: apply exactly what was reviewed, so nothing new slips in between review and apply.
6. Verify: confirm that the resources are healthy and that a fresh plan shows no further changes.

flowchart TD
  A["Change request"] --> B["Generate plan"]
  B --> C["Policy checks"]
  C --> D{"Destroys or replaces resources?"}
  D -->|yes| E["Extra review and approval"]
  D -->|no| F["Standard review"]
  E --> G["Apply saved plan"]
  F --> G
  G --> H["Verify no further changes"]

Trade-offs:

• Strict reviews slow small changes, so automatic approval of low-risk plans, such as tag changes, keeps teams moving.
• A plan can still miss runtime effects, such as a configuration change that restarts a database, so read the provider's documented behaviour for sensitive resources.

In push-based configuration management, a central controller connects to servers and applies changes. In pull-based management, an agent on each server periodically fetches its desired configuration and applies it locally.

When to use which:

• Use push-based tools for ad hoc tasks, smaller fleets and orchestrated sequences, such as a rolling upgrade.
• Use pull-based agents for large fleets that must stay continuously in their declared state, or when servers cannot accept inbound connections.

Pull-based designs also reduce the number of systems that need credentials to reach every server, because each agent only reads its own configuration.

Infrastructure code should refer to secrets, never contain them. The actual values live in a secret store and are fetched at apply time or by the running service.

1. Reference, don't embed: the code names a secret by its path or identifier in a secret store.
2. Generate where possible: let the tool generate random passwords and write them straight into the secret store.
3. Prefer identities over keys: grant a service an identity with permissions instead of giving it a long-lived access key.
4. Protect state and plan output: values read during apply can appear in state files and plans, so encrypt state and mask sensitive outputs.
5. Scan every change: a pipeline check fails any commit that looks like it contains a credential.
6. Rotate regularly: automated rotation limits the damage if a secret leaks.

Trade-offs:

• Fetching secrets at runtime adds a dependency on the secret store, so it needs high availability and caching.
• Encrypted secrets stored in the repository are convenient for small teams, but rotation and access control are harder than with a dedicated store.

The goal is that someone with read access to every repository still cannot obtain a single production credential.

These cloud service models differ in how much of the stack the provider manages and how much you manage. Moving from IaaS towards SaaS gives up control in exchange for less operational work.

When to use which:

• Choose IaaS when you need full control of the operating system or must run software that expects a normal server.
• Choose PaaS or serverless to ship applications quickly with little infrastructure work.
• Choose SaaS when the capability is not a differentiator for your business.

Responsibility never disappears completely: even with SaaS, you manage user access and your data.

A region is a geographic area with its own independent cloud infrastructure. An availability zone is one or more isolated data centres within a region, with separate power, cooling and networking, connected to the other zones by fast, low-latency links.

Design across zones for high availability:

• Run replicas of every service in at least two, ideally three, zones.
• Spread load balancers and databases across zones, with automatic failover.
• Size each zone to handle the load if another zone fails.

Design across regions for disaster recovery and locality:

• Place services near users to lower latency.
• Keep data in the regions required by law.
• Replicate data to a second region and practise failing over.

flowchart TD
  A["Region"] --> B["Zone A"]
  A --> C["Zone B"]
  A --> D["Zone C"]
  B --> E["App replicas and database primary"]
  C --> F["App replicas and database standby"]
  D --> G["App replicas"]

Multi-zone designs are cheap and protect against most failures. Multi-region designs cost far more and add data consistency problems, so they are justified by strict availability targets or regulations.

A layer 4 load balancer routes connections using network information, such as IP addresses and ports, without reading the content. A layer 7 load balancer understands the application protocol, such as HTTP, and can route based on the request itself.

When to use which:

• Use layer 4 for non-HTTP traffic, such as database or message protocols, and for very high throughput with minimal processing.
• Use layer 7 for web traffic, when routing by path or host, inspecting headers, or applying web security rules.

Many setups use both: a layer 4 balancer at the network edge for resilience, passing traffic to layer 7 proxies that do the routing.

DNS turns a host name into an address through a chain of servers, and every answer can be cached for its time to live (TTL). The TTL decides how quickly a change, such as moving a service to new servers, reaches all users.

1. The client asks its resolver, usually provided by the network or a public resolver.
2. The resolver checks its cache: if a non-expired answer exists, it returns it immediately.
3. Otherwise, it asks the root servers, which point to the servers for the top-level domain.
4. Then the top-level domain servers, which point to the domain's authoritative servers.
5. The authoritative server answers with the record and its TTL, and the resolver caches it.

How TTL affects a migration:

• With a long TTL, such as 24 hours, some users keep reaching the old address for up to a day after you change the record.
• Lower the TTL well before the change, at least one old TTL ahead, so caches pick up the short value.
• After the migration settles, raise the TTL again to reduce lookup load and latency.

Keep the old servers running until traffic to them stops, because some clients ignore TTLs and cache answers longer.

Place only internet-facing components in public subnets and everything else in private subnets, spread across availability zones, so internal services and data cannot be reached directly from the internet.

1. Choose an address range for the network large enough for growth, such as a /16 block, which must not overlap networks you may connect later.
2. Public subnets: hold load balancers and outbound gateways, with a route to the internet gateway.
3. Private subnets: hold application servers, reachable only from the load balancer and other internal sources.
4. Data subnets: hold databases, reachable only from the application subnets.
5. Outbound access: private servers reach the internet through a network address translation gateway, which allows outbound traffic only.
6. Repeat each subnet type in every zone for high availability.

flowchart LR
  A["Internet"] --> B["Public subnet: load balancer"]
  B --> C["Private subnet: app servers"]
  C --> D["Data subnet: database"]
  C -->|outbound only| E["Translation gateway"]
  E --> A

Security group or firewall rules then limit each tier to the exact ports it needs, such as the database accepting connections only from the application tier.

The shared responsibility model divides security and operations work between the cloud provider and the customer. The provider secures the underlying infrastructure; the customer secures what they build and store on it. The dividing line moves with the service model.

What this means in practice:

• Many cloud breaches come from customer-side mistakes, such as public storage or over-broad permissions, not from the provider.
• Data, identities and access configuration always remain the customer's responsibility.
• Managed services shift patching work to the provider, which is a strong reason to use them.

Interview tip: give a concrete example, such as the provider patching the hypervisor while you still patch your virtual server's operating system.

A content delivery network keeps copies of content on edge servers near users, so most requests are answered nearby instead of travelling to the origin servers. This cuts latency, reduces origin load and absorbs traffic spikes.

How it works:

• A user's request reaches the nearest edge location.
• If the edge has a fresh copy, it answers immediately: a cache hit.
• If not, it fetches from the origin, returns the response and stores it for later users: a cache miss.
• Cache headers from the origin control how long content stays fresh.

What to cache:

• Static files such as images, stylesheets, scripts and fonts, ideally with versioned file names so they can be cached for a long time.
• Public pages that are the same for every visitor, with a short lifetime.

What not to cache, or cache very carefully:

• Personalised pages and responses that contain user data, which could be served to the wrong person.
• Responses to requests that change data.
• Pages that set session cookies.

A misconfigured cache that stores a logged-in page can leak one user's details to others, so personalised responses should be marked private or no-store.

Cut waste first, then pay less for what you genuinely need, and make cost visible to the teams who create it.

• Right-size: compare real CPU and memory use with what is provisioned, and shrink oversized instances and databases.
• Turn off idle resources: stop development and test environments outside working hours and delete unattached volumes and old snapshots.
• Autoscale: match capacity to load instead of provisioning for the peak all day.
• Commit for steady load: use reserved or committed-use pricing for capacity that runs all the time.
• Use discounted spare capacity: run fault-tolerant batch jobs on interruptible instances.
• Choose storage tiers: move old data to cheaper archive storage with lifecycle rules.
• Watch data transfer: traffic between zones and regions, and out to the internet, is often a hidden cost.
• Tag and report: label resources by team and service, so each team sees and owns its spending.

Protect reliability while doing this: keep enough headroom for traffic spikes and zone failures, and never remove redundancy just because it looks idle.

An autoscaling group adds servers when a metric crosses a high threshold and removes them when it crosses a low one, keeping the count between a minimum and a maximum. Flapping is when the group keeps adding and removing servers because the rules react to normal fluctuations.

How to prevent flapping:

• Leave a gap between thresholds: for example, scale out above 70% CPU and scale in below 30%, never 60% and 55%.
• Require sustained breaches: act only when the metric stays past the threshold for several minutes.
• Use cooldown periods: after a scaling action, wait before the next one, so new servers can start and take load.
• Scale in slowly: remove one server at a time, but add several at once when load jumps.
• Use target tracking: let the autoscaler keep a metric near a target value, which smooths the response.
• Account for warm-up: new servers report high CPU while starting; exclude them from the average until they are ready.

Also remember that scale-in removes a server that may be handling requests, so servers need graceful shutdown and connection draining.

Serverless functions run code on demand, scale automatically and cost nothing when idle. They fit spiky, event-driven, short tasks, but some workloads suit them badly.

Where functions work well:

• Reacting to events such as file uploads, messages on a queue or scheduled triggers.
• Low or unpredictable traffic, where paying per request beats paying for idle servers.
• Glue code between managed services.

A useful check is to compare the monthly cost at the expected request rate with an always-on container, and to measure cold-start latency for the slowest user path.