I sort of understand where this question is coming from, but as someone who's spent a fair amount of time browsing software job postings recently, the real question isn't specialization, it's seniority.
I routinely have found organizations that have, literally, dozens of open listings for "Senior Software Engineer, Sub-field", but few (if any) explicitly non-senior positions available. My very non-scientific estimate would put this ratio at well over 10:1, and probably closer to 20:1 for many of the organizations I've looked through. I actually sort of wonder if they ever take them down, or if demand for senior candidates is just so high that they're basically evergreen listings, where the organizations assume they'll always have positions available for engineers who meet those listings' criteria.
The definition of "Senior" has now changed to "has > 5 years of experience". Combine this with the tendency of people to move every two years (because companies rarely keep up with the market), and you can see that "Senior" casts a wide net. It essentially includes people who may have seen maybe one major iteration of a product, perhaps none. In my opinion, that basically translates to a junior.
As with everything, moving every 2 years has pros and cons. The flip side of what you said is that a person who spent >5 years at the same company has only really seen a limited number of problems, methodologies, technology stack. And that could also count as "junior", right?
I recall this coming up on HN before, but lots of big companies have special programs for hiring jr / entry level engineers. Usually, these are integrated with university and boot camp programs the company has a relationship with. Those roles exist and are being hired for, just through a different process. Another factor is that if you’re cleared to hire someone at a sr salary you can probably hire at a jr salary if you find someone, but probably can’t go the other way. Also keep in mind HR salary bands can go out of date fast in the tech market and you may have to open a sr role in your system to get the range you need to get even an entry/mid level candidate.
My last org just wanted one senior software engineer. Just one. They couldn't find one as the salary sucked, despite interviewing about a dozen over 6 months.
> despite interviewing about a dozen over 6 months
Did they really want one? I'd wager that if they took those six months of interviews, job ads and recruiter time and instead took that money, added it to the salary they are planning on offering (say divided by the average tenure in that role at your company) and threw up a post on the monthly "Who is hiring" thread here they could've found someone.
A common tactic is to say "Oh, we are looking for the right person we just can't find them!" when the rest of the team picks up the workload of the exited employee and to keep that up as long as possible. It happened to me at my first dishwashing job at 13 years old and I've seen it a few dozen times since.
"The explosive growth of the tech sector keeps average age down and depresses average wages. Compared to industries which existed in materially the same form in 1970, we have a stupidly compressed experience spectrum: 5+ years rounds to "senior." This is not a joke." [1]
So companies routinely comb for a "senior" person but the salary is always a mismatch. I have 25+ years under my belt but the companies that approach me expect a 5-year compensation arc and can't understand what the problem is.
Complicated question, as the company was structured so that each layer of management was its own "they" with its own budget. So wasted recruiter time was an externality not on their budgets. Management above line wasn't wasting any time so they did not care. Project accountability rested with the venture leads, not the development teams so there was little urgency to solve those problems either.
There's a good chance these positions never existed.
Company needed to "prove" that no American Engineers are available to fill the position to then justify sponsoring a visa. Then they'll go to the elected officials and whine that there's a "shortage" of engineers to try to get higher immigration quotas for H1Bs.
So they put a junior’s salary and senior experience requirements, then proceed to hire someone with “5 years of experience” at a famous foreign consulting firm (real or not). The experience is then used to justify why new grads have to be paid lower (he’s a senior, we can’t pay new grads the same as a senior) and to qualify for the visa.
They are evergreen listing for my company. They do have some mid-level and entry level, but certainly not as much as senior. We're even known for value shopping new grads to save costs, so I assume the ratio is even worse at other companies.
So - just in case anyone doesn't know this - companies put up listings even if they're not hiring.
Do you know why? 1) Creates an image that they're always hiring and growing (should be self-explanatory) 2) If some holy amazing engineer applies and they can get them for bottom $ - it was basically no cost to get them.
The hardest "engineer" to find "nowadays" is the same as it was 10, 20, 40, 100 years ago: the one who can communicate what [s]he wants to do in a way that non-engineers can grok
And, the flipside of that: the one who can take what [s]he is given in magic, hand-waving, bafflegab statements from non-engineers and convert it into something someone else can actually use
If you can communicate to and understand from others what you want to do and what they want, you're going to get hired
I'm consistently told that communication is my strong point, that I'm good at interpreting statements into wanted functionality, and that my professional opinions are very thorough and well-thought out.
I'm told I'm slow. When I do code screens, usually in a language that I have limited experience in due to my diverse background in technologies, I don't get the job because others are faster.
It's not even reading between the lines. I ace my behavioral and/or conceptual sections. Then I fail the code screen for being too slow. I know they want fast people - they all do. That just leaves people like me in a shitty position, questioning what I should do with my professional life since the first 10 years have been a waste.
I disagree. The hiring process is never interested in your version of the hardest engineer to find.
It's 1. pedigree then 2. "pass this coding challenge" then 3. some soft assessment of how "down for the cause" you are (will you work extremely hard and not quit so easily).
I haven't seen an exception, and I've interviewed widely and been the interviewer at multiple places (given a rubric of what to assess).
I do these technology-based art projects and it is a struggle to explain them.
When it all works people see something small that they relate to, but behind that is a set of practices that aren't obvious and that often I got to by a roundabout route. Frequently I discover these practices instead of invent them and there is a big element of circularity which makes it hard to put together a story that has just the essential elements in the right order, trying to explain things chronologically is just about the worst I can do.
The answer to that is doing it over and over again, doing a better job each time, and occasionally having the breakthrough where you can put things in a new relationship that makes a lot of sense.
If I were you, I'd do write-ups of my process - or at least what I was trying to accomplish, what I did accomplish, what I tried that didn't work, and how I [eventually] got there
That said, you don't need to be able to give your entire life story in 17 minutes flat - but communicating is absolutely the most vital interpersonal skill there is!
That's my belief. At my company we don't have them (or I would be applying to those roles). It's seems most companies know they can save money by combining that with the developer role. I don't see postings for them.
To be honest engineers's earn so much because they're doing multiple jobs. They're also usually doing project management if there's no dedicated project manager.
All too often, the "spec" a BizAn tries to hand over is nothing resembling a "spec", and instead just a bunch of word vomit I have to take back to the original source(s) and ask them to explain everything they wasted hours explaining to the "analyst" because the "analyst" didn't analyze anything - they just reformatted what the enduser(s) said into a standardized form
really curious about if all the people that do leet code can transfer those skills to their job. Problem solving is a good metric i suppose, but how good?
A junior engineer with some (but not too much) experience. Im not sure what exactly has been causing this lately but we have had serious issues at my current org as well as others that I help, finding junior engineers with 2-4 years of relevant experience. We run a fairly straight forward stack (postgres, python/node depending on the service, react front end), our packages are pretty competitive and we have not had a lot of (read basically none) good stuff come across the wire. Recently we have been inundated with resumes from boot camps, college kids, "self taught" people, and just people looking to change careers and expecting us to foot the bill to train them, none of them can hack it on even simple coding challenges or basic architecture questions. At this point ill take someone who can look at a common stack dump and have a fair idea of whats going wrong...
The best analogy I can think of is that we dont need Frank Gehry, but we need someone know knows the studs should be 16 inches apart, and all we are getting is people who know that 3 inch screws might be involved in the job but that is the only screw they have ever seen and the only screw they are comfortable using.
Is the issue, possibly, that virtually every company has decided that hiring people and training them up in the requisite skillsets is an unreasonable ask? Despite that being the pattern for employment for most of the history of employment as a concept? Where do you think the population of junior engineers is going to come from if everyone thinks it's unreasonable to "foot the bill to train them"?
Drives me crazy that people don't want to hire entry level engineers, don't want to train people, and then complain they can't hire anyone. I was a bootcamp grad, and I was pretty useless for at least the first 6 months after I got my first job, as are most entry level engineers. Since then, I've been a pretty substantial bargain for every company I've worked for, based on the fact that I've managed 25+% raises at each subsequent job hop. I'd still be working at the first place I ever worked if they'd just kept my comp roughly in the same ballpark as what I was being offered.
Almost none of the more common ones are hard to find for the right price. The issue is that no one wants to pay.
I have friends trying to do some computer/robot vision stuff. They need people who specialize in that field. It's a bit niche but they know plenty of people. You can imagine someone with that expertise likely has a PhD and has worked in the field for some time. Problem is: They don't come cheap in SV because that person could easily get a job doing something similar at a huge company with a big offer. So, you're going to struggle as a startup who is trying to pay very low $.
And if they didn't do some machine vision stuff then they'd easily be able to transfer to some typical role. So, therefore, you gotta still pay top dollar... It's a real issue for folks who can't raise huge rounds.
How do you differentiate these recruiters from the run of the mill spambot recruiters? Or is this a delusional comment by someone who didn’t realize they were being sold?
They typically researched you somehow, they know the company you're at, and likely previous ones. They tend to approach you as a conversation about what you want to do, and what the company needs and try to find common ground that's good for both of you.
It's not just a bullet list of requirements and benefits.
In my observation, Data Engineering leadership is the hardest skill to find recently. There is a good amount of entry-mid level individual contributors in the market, but at the Manager+ level the market is insanely competitive .
I find that the alphabet soup of technologies they use on the DE/ETL/warehouse side of things is so alien it's like they come from an alternate timeline. That alone narrows the field considerably.
I dont think there is any one type of engineer that is hard to find, rather the difficulty is finding them within your company's budget. There is a lot of talent out there, it just doesn't come cheap. The catch is that the companies that have to the budget to lock up top talent generally have large bureaucratic departments where their best engineers are really bored and/or working only 3 hours a day, and using the job to fund their lifestyle while they launch their side project.
Security Engineers that know how to secure applications running on a cloud platform are rare as hen's teeth. We've tried and failed to hire one for almost two years now.
All the candidates are either "cloud security experts" who will run through a checklist of AWS best practices while remaining wilfully ignorant of the application itself, or on-prem dinosaurs who want to talk to us about the ports on our corporate firewall.
Hiring a person and training them for the role is going to be quite a lot cheaper than finding that rare professional, who's going to command a very high premium.
Which means that at least one of the following things is true:
1. You're not willing to pay enough for an individual who's a perfect fit.
2. You're not willing to hire someone who's not a perfect fit and spend money to train them for the role.
3. You've done the math and concluded that the cost of doing either 1. or 2. above is higher than the value brought by actually securing these applications (who's doing it now? nobody?)
So in cases 1. or 2. it's entirely your company's fault, and in case 3. it's nobody's fault and you don't actually care about the end result. It's an evergreen listing for a job that you already decided shouldn't exist.
It's very difficult to train somebody for a competency that's lacking in the organization. Could you train up a DBA or React expert if you had no expertise in those subjects?
I didn't claim it was anyone's "fault" - OP asked what type of engineer is hard to find, and I answered. Obviously #3 is the real truth, which is why IT security is and will continue to be a nightmare
I think they are rare to find since corporates mostly prefer checkbox insecurity and not a well rounded security professional, better to grow your own in-house talent.
I worked one place with competent DevOps, enough app-level holes to drain spaghetti, and willful ignorance from leadership.
Compliance is a joke, and real security is for companies who deal in it — they get to have the security experts, and companies get some security by chance.
Agreed with the senior engineer sentiment. Finding entry level is difficult, but doable. Especially with a good intern program, etc.
But finding _any_ senior engineers is hard right now. With FAANG paying so much, it's just hard to compete.
IMO it gets even worse when you end up in specialized ares. Eg CV, ML, etc. Pretty much anything that requires a masters degree (or a heck of a lot of self motivation)
Technical skills are only part of the equation. You personality is more important. If you are a perfectionist then you are a problem. If you are high maintenance then you are a problem. If you can't work with the team then you are a problem. All these things are just as important to finding the right developer.
In my (very limited) experience, small companies that separate out SRE and devops are a bad smell when it comes to type of work. It ends up being something like:
Devops: Someone who is oncall for fixing all pipelines and making sure everyone has access/credentials for various services (AWS, git, etc). Maybe if they have time they can start to teraform things.
SRE: Someone who is just oncall for everything and doesn't have time to improve things
On the other hand when a job listing is not specific but mentions SRE/Devops, they seem to be looking for someone who can build new systems to make things better, rather than just keeping the lights on.
They are related but separate. SRE Handbook for Google targets that an SRE spends ~50% of their time to Dev Ops work. It wouldn't be unusual for DevOps to turn into an SRE or vice-verse.
Am a self starting generalist. Every company I've worked for has desperately tried to beat it out of me. They think they want this but what they actually want is someone who will turn pre-chewed tickets into code on whatever platform they have without attempting to fix underlying issues as they identify them.
I used to think like this, but what I ultimately realized was that it's pretty arrogant for an individual contributor to think they know best what the user wants.
Once I realized that, it made a lot more sense why I wasn't (and shouldn't!) be the one calling the shots on priority, UX, tempo. Even for things like tech debt, it really needs to be baked into the ability to deliver new value to a customer or it's very often not worth addressing.
Ultimately, I tried to get closer and closer to the users, so I could be in a position to know what they want, but as I did that, I found myself coding less and less.
It's a tradeoff. If you really love to code, you must accept you need help figuring out what to code. If you would rather build things people like, you must accept that means spending a lot of your time talking to and working with users.
To clarify, I work embedded systems for aerospace so it's a very different world than most devs exist in. The customer is asking for something defined in a legal contract with the details spelled out for the most part. The issue is the industry is inhabited by dinosaurs so the older I get, the more I realize that *nobody* knows what they're doing. Everyone is just trying to maintain their silo so they can keep getting a paycheck. As such everyone is still using c/gcc/svn/eclipse/windows/vxworks and if you even think about changing any one of those things in the slightest you'll be inviting a firestorm of pushback as it threatens to break long established and scripted personal workflows that only the individual understands: https://xkcd.com/1172/
Corporate almost always wants faster/better/more agile/higher reuse and you see that in postings/interviews but the PMs/Principle types always dig a moat to prevent anything from moving down to Seniors and lower.
You can't be a one man band, but if the organisation supports it, you can achieve both. In my experience, it doesn't need to be a trade off. We can get close to the users and program to your hearts content. We need the right company and environment to want to invest in that for us. There's a great loop we can get into by doing things like offering demos of new features to clients weekly or bi-weekly. What I've seen happen is that we don't need to code as much because we know the specific problems that need to be solved, rather than loads of guesswork and trying to write blanket solutions for misunderstood problems.
If that's the route you want to go, prepare to spend more than half of your time not coding.
I personally think this is the future, and the myth of the introverted software developer who comes up for air to say 5 words in a standup is well past its expiration date, but a lot of people cling to that pretty strongly, so I expect people will prefer to code in dysfunctional teams that aren't well connected to their users for years to come.
What a luxury to work at a place where tickets are defined and work is scoped out for you :)
If you're chafing there due to the lack of autonomy, I promise there are places hiring people whose strong suit is tackling ambiguity in addition to software development.
No deadlines, nearly no meetings, and definitely no pre-chewed tickets. Write your own spec and then ship it (all with help from team mates ofc).
I'm pretty confident that we're not the only company that works like this.
What I think I mean is, if you're appalled by employers that treat people like replaceable cogs, then it might be worth your while to find an employer that doesn't :-)
I used to work in embedded software too. At the risk of sounding arrogant, it is my experience that embedded devs learn the web stack a lot faster & more thoroughly than vice versa. So should you at any point be interested in switching to web/mobile tech then be sure to get in touch.
And, totally unrelated, when I did embedded software, by far the coolest job I had was at a hardware startup. We were 9 people in a room making a lithography machine (a machine that makes chips) that could compete with the market leader's 20 year old junk. Said old junk was getting, well, old and broken, so we were trying to fill that demand gap.
I think at least half the team was working well outside their usual skillset at any point in time. It was super frustrating for people who considered themselves specialists and awesome for the rest. I learned a lot there about what a workplace where generalists thrive looks like.
In my experience: the FE engineering fields (web FE, Android, iOS) are probably (very slightly) less in demand than most others. I think that's primarily because: the technology has calcified (in a good way), and there's tons of bootcamps that churn out pretty high quality associate-level engineers which specialize in the FE world.
These bootcamps will also advertise as having covered backend stuff, but its very rarely in any level of depth beyond "spun up an express NodeJS app as a razor-thin layer in front of Firebase" or something; thirty minutes of interviewing will expose this gap, and they get hired on as a FE.
BE/DevOps is in higher demand, thus harder to find. DevOps is a weird one, because its a field that's extremely striated between platforms. Whereas a Java BE engineer could ramp up on NodeJS or whatever very quickly, an Azure specialist will take longer to match their skillset to AWS, not to mention the large body of individuals coming in with traditional linux server maintenance experience that will have a hard time matching their skillset to cloud-native serverless-like roles. It shouldn't stop a hire, but it does make finding well-matched qualified candidates more difficult.
Actually, I'll give props to a lot of the modern Azure skills DevOps peeps can develop; they're very big on their state-of-art being open source CNCF stuff; ex, Kubernetes, DAPR, KEDA, etc. So, a skillset there can transfer to a ton of places (including a GCP-esque shop, or even an AWS shop on less managed tooling). Glad to see some modernization/standardization in this space, but AWS is very antiquated and stuck in their ways.
Less from a hiring manager angle and more from an engineer angle: bonafide actual full stack devs are insanely valuable. Every once in a while you get a candidate that can apply to any role, but will apply to a specialized role; then they meet the engineering manager and the manager realizes, they're a candidate among a pool of candidates who are specialized enough for the role, but they also bring that full stack experience. Very few teams/roles are actually "100% backend" or "100% whatever"; the team may settle for someone who meets the job description, but will prefer breadth over depth, in nearly all cases.
Rarest quality: seniority. Not skill based. Just time, and more-so a demonstrable breadth of experience on a handful of projects, tech stacks, etc. Can't fake it. Can't stress how many roles we've posted at a Senior level, then settled for someone more mid-level because there's no-one. We're remote; competitive salary; small team; cool tech; cool product; maybe one in fifty candidates feels demonstrably senior.
What could someone applying do to prove their skills? Is submitting a couple of GitHub repos enough? Or do you just not stand a chance without at least a degree?
I routinely have found organizations that have, literally, dozens of open listings for "Senior Software Engineer, Sub-field", but few (if any) explicitly non-senior positions available. My very non-scientific estimate would put this ratio at well over 10:1, and probably closer to 20:1 for many of the organizations I've looked through. I actually sort of wonder if they ever take them down, or if demand for senior candidates is just so high that they're basically evergreen listings, where the organizations assume they'll always have positions available for engineers who meet those listings' criteria.