Crazy DevOps Interview Questions (4)

Note: This is the fourth article of the series. The others are: part 1, part 2 and part 3.

You have a shared document open when the phone rings. The interviewer, speaking with a thick accent, asks:

“What does ls * do?”

It sounds far too easy to be true. You say it lists all the files in the current directory. The interviewer asks a couple of follow-ups about how it works, and you explain that the star is passed to ls as an argument, which somehow makes the binary walk the whole directory and print its contents. Simple!

The interviewer thanks you and hangs up, and you’re left grinning. That lasts until the recruiter calls to suggest you try again in a year or two.

So what actually happens when you run it in a Linux shell? You’d see something like this:

$ ls *
1file 2file 3file

4dir:
a b c d

5file 6file

7dir:
e f g h

8file 9file

Notice that ls descended one level into the subdirectories. That would be impossible if it only listed the current directory, so something else must be going on.

There is no magic here. The shell expands the star into a list of matching files and directories before ls runs, then hands that list to ls as arguments. ls then processes each argument in turn, listing files as they are and showing the contents of directories. This is the “trick” the interviewer is after, but a few more details are worth mentioning:

• Hidden names are skipped. The star expands only to names that don’t begin with a dot. Dot-files aren’t truly invisible. By convention, dot-prefixed names are reserved for local configuration files.
• The argument list has a size limit. If the expanded list has too many entries or is too long, you get a “too many files” style error. The limit is high and varies between systems, but it is finite. You could hit it if, say, a program writes status files into a single directory for a long time. The GNU Core Utilities FAQ explains this in more detail.
• Empty directories behave differently. If there is nothing for the star to match, the shell leaves it alone and passes a literal * to ls, which then reports a “not found” error.

If you answered along these lines, expect follow-up questions about handling “too many files” errors. These are usually solved with find, which brings us to the next question.

The interviewer presents this scenario:

“I tried to delete everything in the current folder with rm -f * and got an ‘argument list too long’ error. I really need those files gone.”

As noted above, find is the right tool because it enumerates files one at a time instead of building one giant argument list. For example:

$ find . -type f -print

On its own this not very useful, but it is easy to turn into a scalable equivalent of rm -f *:

$ find . -maxdepth 1 -type f -delete

If you’d rather call rm explicitly:

$ find . -maxdepth 1 -type f -exec rm -f {} \;

Or you can pipe the results to xargs:

$ find . -maxdepth 1 -type f -print0 | xargs -0 rm -f

Note: -print0 separates file names with a null character, and xargs -0 reads them back in that format. This is necessary for file names containing unusual characters such as spaces, tabs or newlines.

Having solved a hard problem, you get an easier one to finish the interview: rename every .htm file in a directory tree to .html. Sounds easy, right?

Your first instinct is correct: use find to locate the files.

$ find . -type f -name "*.htm" -print

Now what? You could pipe the output into a script, but the interviewer probably wants a one-liner. Here is the simplest approach:

$ find . -type f -name "*.htm" -exec mv -f {} {}l \;

Did you spot the trick? find replaces {} with the current file name, so {}l is the same name with an l appended.

The follow-up is harder: do the reverse and rename every .html file to .htm. This calls for some shell string manipulation using the % operator:

$ find . -type f -name "*.html" -exec sh -c 'f="{}"; mv -f $f ${f%l}' \;

Notes on the command above:

• -exec launches a shell with an inline script, and find substitutes {} with the file name.

• The script is in single quotes so the calling shell doesn’t expand $f before the inline script runs. Without them, the assignment just before it would effectively be ignored. This is a tricky point. Try the following and compare what gets printed:

  $ a="outside"
  $ sh -c 'a="inside"; echo $a'
  $ sh -c "a=\"inside\"; echo $a"
  

• ${f%l} looks odd, but it simply strips the trailing l from the value stored in $f.

I’m curious how many people could solve this in an interview without having worked through all these pitfalls first. Not many, I suspect.

That’s all for today. Thanks for reading!