Crazy DevOps Interview Questions (2)

Note: This is the second article of the series. The others are: part 1, part 3 and part 4.

Imagine you run these commands:

# cd /tmp
# mkdir a
# cd a
# mkdir b
# cd b
# ln /tmp/a a

What happens?

A candidate might note that this hardlink would create a circular reference. That’s right, but it’s only part of the story. How would the operating system or file system actually deal with the command?

A command-line expert might brush the question off by saying hardlinks to directories aren’t allowed. Another might notice that ln is missing the -d option, so the command fails before anything else. Both are correct, but neither is the full answer an interviewer is looking for.

A complete answer should cover the following:

• Most file systems forbid directory hardlinks, but not all. The best-known exception is HFS+ (Apple OS X).
• By definition, hard links are directory entries that point to the same inode. The inode holds a link counter, and deleting a hard link only removes the file itself once that counter reaches 1.
• Directory hard links aren’t inherently dangerous, so that’s not why they’re disallowed. The real problem is the circular reference described above. Graph theory can solve it, but doing so takes a lot of CPU and memory.
• The decision to forbid directory hard links was made because of that cost, which grows as the file system gets larger.

A thorough answer like this is what you’d typically expect in an interview at one of the “Big Four” technology companies.

Suppose you run this command on Linux:

# mv /home/testuser/magicalapp /usr/local/

What does it do, and how does it work? If it doesn’t work, why not?

The answer comes down to how mv basically works. It doesn’t necessarily move any data. Most of the time it just creates a new directory entry at the destination that points to the same inode as the source, and the source entry is removed in the process.

When the source and destination are on different file systems, mv generally behaves like this:

# cp -r /home/testuser/magicalapp /usr/local/
# rm -rf /home/testuser/magicalapp

Note 1: Very old versions of mv couldn’t move directories between file systems.

Note 2: The PowerShell equivalent (move item) works differently. It can move files across file systems but has no recursion option.

It’s a good interview question, don’t you think? I’m not sure how many candidates would get it right.

What does each field in the CPU usage line of top mean (for example us, sy, wa)?

The general answer is that these fields are percentages that add up to 100. Each one shows the share of CPU time spent on a particular kind of work:

• us: user-space programs, meaning ordinary processing that isn’t otherwise categorized.
• sy: kernel-mode processing, i.e. everything the kernel does.
• ni: user-space programs running with a modified priority via nice.
• id: idle time, when the CPU effectively runs NOP instructions.
• wa: time spent waiting on a resource such as a disk or the network. It doesn’t mean busy-waiting. It’s CPU time spent sending or receiving raw data to or from external devices.
• hi: time spent servicing hardware interrupts raised by external devices, such as a physical network card. A high value here can point to a DDoS against the network interface.
• si: time spent servicing software interrupts. User programs ask the kernel for services through software interrupts, though there can be other uses as well.
• st: stolen time (not applicable on physical machines). This is CPU time taken by the hypervisor in a virtualized environment.

That wraps up the second installment of the interview questions. I hope you enjoyed it!