How Much Should A Software Engineer Know About Operating Systems?
This weekend I was reading Operating System Concepts, 9th Edition, commonly known as 'The Dinosaur Book'. It seems to be well regarded as THE book on operating systems for undergraduates. I've been reading it maniacally since I have an open-book test for my OS class on Monday, and the course is entirely based on this book.
I've been really enjoying reading it (and not in a Stockholm syndrome kind of way), but it got me thinking about how many of my class mates will NOT be enjoying reading it, and probably aren't reading it at all. This train of thought led me to wondering about how much of this knowledge the average software engineer (or developer) has.
Do they know about preemptive vs nonpreemptive process and thread scheduling? Do they know their page replacement algorithms, or how modern operating systems make use of virtual memory? How often do they consider thrashing while developing? Do they need to?
These are questions that I don't have the answers to, as I most certainly don't have enough experience interacting with the 'average' engineer in my area. Not to mention the higher-tier areas where everyone is a super-engineer.
A couple months ago there was a guest on Scott Hanselman's HanselMinutes podcast who mentioned that he thought every developer should have a working knowledge of the 'level below' their regular work, and AT LEAST be aware of the issues of the level below that. If my memory serves me right, by levels he meant how abstracted the level of work is from the bare metal of the machine.
In the podcast they were talking about JavaScript libraries, but I think the issue is well demonstrated by the Java language. In this land of levels, a Java developer will obviously be very knowledgeable about the Java programming language, but they should have a working knowledge of the Java Virtual Machine (JVM) and JVM Bytecode (what happens to my code when I compile it), and they should be AWARE of how that level interacts with the one below it. How does the JVM interact with the operating system it's running on? What can, and does, it do?
I think the vast majority of developers will find that the operating system is either the level directly below their level of work (and thus should have a working knowledge of it), or the level below that (and thus should be aware of what it concerns). This makes sense since I think you'd be hard pressed to find a developer that wasn't at least somewhat aware of process scheduling and possibly parallel processing.
I'm interested in what you, the reader, have to say about this though. Have you read Operating System Concepts? Did you find it useful? How much do you know about operating systems, and how much would you say your colleagues know?
Check out my other blog posts! If you found this post interesting, feel free to let me know either on Twitter (@Isaac_M_Jordan), or in the comments section below.
Enjoyed my post? Sign up to the newsletter to receive a small email when I post. No spam, I promise.
-
Brennan W. FIeck on Feb. 10, 2016, 5:08 p.m.
I don't think a Java developer needs to have knowledge about anything deeper than Java code, and here's why: that's not what Java is for. For basically any other language optimizations require a working knowledge of how your tools work and what the system is capable of doing. If you're writing C, you need to know about how different OS's interact with user-space binaries, if you're writing html/css you need to know how the different major browsers will render it, etc. Java however, is a language based on the concept that you dump the workload of such problems on the compiler. Java is meant to quickly produce software that will run on a myriad of systems with minimal localization, if any. I once asked a Google representative why they use Java, even though it's slow compared to C, and even Python, and he said "because it allows us to develop very quickly without worrying about performance". So for basically anything but Java, yeah you should absolutely have at least cursory knowledge of the system on which it runs, but specifically for Java or languages like it, no by design.
-
Bob Saget on Feb. 10, 2016, 7:57 p.m.
Sure, you don't need to know the internals of Java and how it interacts with the OS for doing simple things that only concern Java. Knowing how the compiler and JITter will behave can save you a few cycles here or there. But fairly often you need to start interfacing with other components that are not as isolated as Java, or doing things with threading, or pushing tons of data around. When you start doing things like that the more you know about how the OS and execution environment tend to behave the better off you are.
Google has the luxury of being a company with tremendous resources and has written the vast bulk of it's software within ten years. When you're working outside of Silicon Valley, your clients/employers tend to have components in their system that are very old. Knowledge of the underlying OS will be very helpful at some points in your career, but not all the time.
-
-
Mike G on Feb. 10, 2016, 8:18 p.m.
Depends on your job. Different projects will require different levels of knowledge. A colleges job is to give you a broad overview of the basics so that you can fit almost anywhere at an entry level. Depending on what job you do, you may or may not need everything you learned in college. I have a good overview of Windows, I know a little about Linux, and ovoid learning Mac. I have also coded for all 3 platforms successfully.
-
halukag on Feb. 11, 2016, 9:31 a.m.
As much as millions of drivers need to know the inner workings and detailed design of the engines of their cars...
-
C Wat on Feb. 11, 2016, 4:31 p.m.
I don't think this is a valid comparison. Software engineers are more like mechanics, not average users, so require a more in-depth knowledge.
-
Isaac Jordan on Feb. 12, 2016, 9:39 a.m.
Bob Saget covered the points about interaction in Java, but I thought I'd say something about the performance of Java against C and Python. I would say that it's a misconception that newer versions of Java are sluggish or slow. The performance can quite easily beat Python if you assume competent programmers in both languages. As for C, sure it's going to be probably at least 20% faster most of the time (including JVM start-up time), but it's really quite rare for a 20% performance difference to really matter in application development. Speed of coding, and cost of maintenance are really much more important there (obviously performance is to a certain level as well though).
-
ai_jarvis on Feb. 9, 2016, 11:43 p.m.
I do believe that more developers need to at least have a cursory knowledge of how their code, execution environment, and OS interact. I was at a company that spent several years and lots of time/money trying to track down an issue that everyone on the hardware side said was software and on the software side was hardware. When my turn came up in the on-call rotation to deal with this issue I was able to look at data that described things like CPU load, RAM utilization, etc. and see something was amiss. What it ended up being what I think is a small change in how the VB6 execution environment interacted with the post NT kernel. Long story short, what used to work well as 'sleep' function in the code was no longer acting like a 'sleep' function. When that was corrected, in the code, everything cooled down and the failures completely stopped.
Isaac Jordan on Feb. 9, 2016, 11:53 p.m.
That type of bug has to be my current favourite. Having to dive right down to lower-levels, and find out that something logical is causing this bug - and then being able to solve it gives a great level of satisfaction.
A couple days ago I was writing some C for my Networked Systems class, and for some reason my buffer would occasionally get duplicate content if I refreshed fast enough. I tracked the problem down the a malloc that should have been a calloc (to clear it) since it seems that the character array stored in that memory location was sometimes left over from the previous response. That bug was very satisfying to fix, and it was only because I knew there was a reasonable chance the OS would allocate RAM in the same location as before.
Thanks for sharing your story!