This is the third profile in a series on Linux Foundation system administrators leading up to SysAdmin Day. Do you have a super-hero sysadmin you’d like to recognize? Send your nomination to This e-mail address is being protected from spambots. You need JavaScript enabled to view it
by July 25 and enter them to win a free ticket to LinuxCon and CloudOpen North America taking place in Chicago August 20-22, 2014. (See the full contest announcement for more details.)
Michael Halstead maintains all of the public facing infrastructure for the Yocto Project, a Linux Foundation collaborative project that provides the tools and methods for building custom embedded Linux distributions. In this Q&A he describes his typical day at work, the best part of his job, how he spends his free time, and more.
Linux.com: How long have you been a sys admin?
Michael Halstead: It’s hard to say. I’ve been administering systems since I was 15. I was the only student at my high school interested in taking over responsibility for the Red Hat systems from the student admin who was graduating that year. No one on staff had any idea how to run these systems, but our school of nearly 1,000 students depended on them for classwork, calendaring, and announcements. I was first paid to administer systems at 19 for a small health research non-profit. My job title switched from Senior Technician, to Developer, to Web Admin, but I never stopped having server responsibilities. It wasn’t until I was 30 that I actually had the job title – Systems Administrator. I’ve always loved my servers more than my code, and I’m very happy to have specialized.
When did you start at the Linux Foundation and how did you get the job?
I’ve been with Linux Foundation since January 1, 2012. Several active Yocto Project users knew they wanted to work with me. They approached the Linux Foundation and said, “Please hire Michael.” It was pretty simple and super exciting at the same time.
What do you do for the Linux Foundation? What’s your specialty?
I maintain all of the Yocto Project’s public facing infrastructure, including Yocto Autobuilders, which are based on Buildbot. I have odd specialties – such as storage and LDAP. I also used to be a Drupal developer, and will often dive in when CMS questions come up.
Will you describe a typical day at work for you?
Wake up and read e-mail, then skim the logs. Answer e-mail. Prioritize problems. Start fixing. If everything is working correctly I have several long-term projects, or low priority requests to take care of. Sometimes I’ll spend a few hours reading about OpenStack, Docker.io, or the latest hybrid volume technology to stay informed. If I’m feeling lonely, I’ll go answer questions on the mailing list, or peek at IRC.
What’s your favorite part of the job and why?
My favorite feeling is when communication just flows. Everyone is talking and typing, and nobody is interrupting. No messages get missed. When all the lines are open I can unblock people and make their day better. Our work requires a lot of quiet, focused time, so most days move at a slower thoughtful pace. When we are really chatty and moving quickly I find out about small annoyances, normally too little to mention, and I can remove them. I love making people happier about the systems they use every day.
What is your nightmare scenario? How have you prepared for it?
We have a 90TB NAS device at the Open Source Lab in Corvallis, OR. Waking up to find it offline on a holiday week would be the worst. We only have access during normal working hours so it might be days before I could get in and see what happened. I’d have to tweak the build system to keep builds running, and they would be slow. We’d lose access to our shared state files, local source mirrors, and our nightly builds for the last two months.
We’ve prepared by building in lots of redundancy. There is no reason it should go offline without plenty of advanced warning. And even if it does, we can rebuild any data there from source. We also have offsite backups of important data. Still, it’s [something I worry about].
What is your favorite sysadmin tool and how do you use it?
Today I’m going to say Wireshark. When services have really weird problems, you have to be able to see what’s 
actually happening on your network, and Wireshark is great.
What’s your favorite story about working at the Linux Foundation?
It’s more a feeling than a story. Everyone I work with is so good, that they aren’t afraid. Other groups worry about new technologies and it shows in their decision-making by limiting what software or distributions are allowed. Here at LF we have our preferred tools, but if someone on the team wants to test something new, nobody worries. No one is concerned, “Will they be able to figure it out?” Of course, we can learn, if it turns out to be a better approach, we’ll use it. Have you ever worked with someone who refused to use new software until they had been trained? I find that frustrating. No one here is like that. We have broad foundations of knowledge to work from. We can all read documentation, learn as we go, and adapt to changes as they come. That’s the best.
What do you do for fun, in your spare time?
I go in the woods to see the plants and animals. I visit vineyards. I learn recipes. I read about games, and stories. I attend conferences and their after-parties. I watch plays. I flash a new OS on my router or phone. I float down rivers. And I brew beer for my friends’ weddings.
Read more about Linux Foundation System Administrators:
Linux Foundation SysAdmin Andy Grimberg Loves New Tech and Snowboarding
To Linux Foundation SysAdmin Ryan Day, Elegance is the Best Tool