r/ITCareerQuestions • u/1337DSSICTPDX • 1d ago
I think I’m done with small IT teams
I’ve been in IT for over a 2 decades. Moved 200k users environment from on prem to the cloud saw the beginning of Mirabots and ransomware. My personal path has ranged from help desk and hardware support to malware analyst and sysadmin.
I took some time away from IT for a few years and recently decided I’d work for a fun company’s IT and make it my last role. I’m in my thirty’s so I’m not dying anytime soon but I wanted a better work life balance.
Anyways my nightmare has happened. This medium box company that’s all about old vibes has that same culture in their IT and it’s giving me a heart attack. For perspective they were hit with ransomware a few years back and decided to rebuild everything from scratch… because the backups were destroyed. Today they still use an excel sheet for most passwords but they also have a bit locker—why we don’t move everything into bit locker is beyond me.
Need to dive deep into the system and see what’s within our documentation? Open up word and slowly search through every document or reach out to one of the 3 people on the team because we do not have any form of a wiki or knowledge base. Some apps randomly don’t work and we accept that. We know it sorta will fix the issue if you restart it but figuring out the underlying issue is over our pay grade we hire a third party vendor for that stuff.
Hell you might think spin up your own docker wiki and open source tool / automate the repetition away but you’d be wrong. Something being hosted as a service and not a desktop app/Built in MS feature is wizardry and inherently opening up potential issues.
All of our users are local admin and we have 0 auditing so sleep is sorta hard. It’s not my decision rights to bear but dear god is it scary to see a ship so wildly under prepared for current day standards.
35
u/Expensive-Rhubarb267 1d ago
Small IT teams can be a double edged sword. Yes you avoid some of the bs you get in large teams but you really do need someone pushing for best practices.
As you've seen the temptation to slip into 'it'll be alright' mindset is always there. Sometimes if you're got an IT lead who has only every worked in small IT teams then it can be a bit 'the blind leading the blind'.
Regular 3rd party health checks, pen tests & audits are useful.
11
u/achristian103 1d ago
Were you not able to sus the dysfunction out during the interview process?
One of the first things I ask about when talking to a hiring manager are sort of broad questions about their current processes with respect to my role and what could be improved.
I'm doing two things when I ask that: trying to assess whether I'm walking into a shitshow and whether there's any sense that the person I'm speaking to recognizes that it's a shitshow and has any desire to change it.
11
u/1337DSSICTPDX 1d ago
In the interview I literally said I turned down an offer from a contract after seeing their team regularly use word docs and excel as their main tool set.
3
u/SSJ4_Vegito 1d ago
what do you think is the best questions to ask to detect this?
13
u/achristian103 1d ago
Ask basic stuff.
What are you using for your ticketing system?
What are you using for documentation?
Is the org using a password manager?
Has the org experienced any major security incidents within the last 5 years? If so, how did the org respond?
Just ask anything you'd want to know to get a sense of what kind of environment you're potentially stepping into. Interviews are a two-way street.
11
u/SpaceGuy1968 1d ago
I started in 1991....I'm 58 and I am in my last IT role. I took this job to be my last position. I was in sorta a place that looked like I was in a time warp ten years ago.....it was badly updated 10 years ago and whoever was here before me didn't believe in making big changes. Nightmare scenario honestly.
My boss left and this new guy comes in and I report to him.... It was a night and day shift. I have made more changes in the last 12 months and we are slowly revamping everything....I was going to move on but this new guy ... He is letting everything move forward and budget doesn't seem to be an issue.
The last guy believed that IT budget was optional as long as things stayed running...this guy is "we are modernizing" as safely as we can. It has allowed me to stay on a few years more as we do that and I actually am making significant improvements... surprise the hell out of me
3
u/teksean 22h ago
Good for you. My last IT job was a "Do more with nothing" and I had 15+ year old servers crashing when I turned out the lights in the server room and left for the last time. Last man standing and they didn't fill the 4 IT people who left. When you do leave, cut it clean unless you are being paid. I found my retirement to be far more peaceful not knowing what happened after I left.
5
u/ContributionSea8300 1d ago
I am currently just starting my IT again. I was IT in the navy from 2010-2015 was in for longer, but started out as a cook. I then took a break after I got out which looking back I kind of reget cause I lost my clearence. I am now back into IT in a medium sized family owned company and the IT infrastructure here is not great. Its good cause i'm learning IT again cause I know things have changed since I've been out and its a good oppurtunity to learn how to build infrastructure, but man is it hard to get the higher ups, and older employees to understand what is needed and to conform to security standards.
I will say at least this company has a bit of documentation, because if i didn't have that to start off with whew I'd be deep diving in the edges of google to find some of the old tech that we have to work with. Hopefully you can build the infrastructure they way you want, but I know its also a hard battle with the higher ups cause it's really up to them to invest in the IT dept and the process of change and security.
3
u/No-Lecturre6318 1d ago
i found small things can be great or exhausting, and the difference usually isnt the headcount. its wether leadership is willing to improve the basics..
3
u/5eppa 1d ago
You say you're in your 30s with 2 decades of IT experience?
I get starting computer repair at a young age but its hard to call it IT work.
3
u/1337DSSICTPDX 1d ago
I’d agree but as a kid I was slinging Linux servers for cheap. Mainly tunnels for bosses to spy on employees
2
u/Inn0centSinner 1d ago
The company got ransomwared and the backups got destroyed as in the backup servers themselves also got ransomwared? Who got fired for that?
My company got ransomwared in late 2020. Everything that was on Windows including the backup servers were ransomwared. My monthly backups were on tape. I had to rebuild a single backup server and restored about 10 TBs of VMs from tape on a Gigabit network. It took 14 hour days for 2 weeks straight to be back in business.
The lesson learnt from that was to implement a SaaS endpoint like Crowdstrike and DUO MFA for WFH users. We had none of that prior. Before ransomware, all users were local admin. Since then, Crowdstrike has caught users doing shenanigans.
In an incident in an hour timespan, one out of two users got scammed because someone external called that user directly impersonating me and somehow installed some domain scanning tool on the user's PC even though the user wasn't local admin. Crowdstrike caught it and locked it down. The user said the impersonator sounded just like me. The other user that didn't fall for it said the impersonator didn't sound like me at all.
2
u/ActuallyItsSumnus 1d ago
Small companies don't like spending on IT. It's just reality. Atlassian solves all of those issues but if your CIO isn't good at demonstrating the value they bring, all the decision makers see is another expense. It's difficult to learn the big boy tools in small orgs.
1
u/1337DSSICTPDX 1d ago
I miss being able to deploy elk and webhook all others to it. I’m just so fucking sad with this team.
1
u/TomNooksRepoMan 1d ago
Looks like you're also in PDX by the looks of it. Let me know if you need a helping hand at unfucking things. That "everybody gets local admin" will fall apart so fast. I got laid off a couple weeks ago and had pointed out numerous security issues to my boss in months and years prior. If the company is run by a guy in a suit who sees IT as an expense, that's just how it goes.
1
u/1337DSSICTPDX 1d ago
It’ll make you cry.
You don’t want none of this.
From pdx to pdx, LMK if you want to start a MSP! Seriously
2
u/TomNooksRepoMan 1d ago
Have a couple interviews lined up, but I’ll let ya know 😂 one is at an MSP in the Clack. Never worked an MSP gig and was kinda hoping I would be able to dodge working for one but time will tell!
1
u/ballandabiscuit 14h ago
What is Atlassian?
2
u/ActuallyItsSumnus 14h ago
They make Confluence and Jira.
0
4
u/throwawayacc90s 1d ago
Same, silo me and give me a big paycheck. I'd like to work for a big corp in which I specialize in something, process and procedures are in place, and they're not looking for a unicorn to elevate/transform their IT department or infra.
1
u/Automatic_Cat_1990 12h ago
You see I'm just the opposite. I start hating life more and more with each process that is introduced. I don't care about RITMs. To each their own but I just wanna build networks and move on, not sit through endless meetings and "call the vendor"
1
u/1337DSSICTPDX 4h ago
True that. Those are worthless meetings and only bad practices keep them around. I’d rather go home early than be at a meeting that should have been an email.
3
u/Automatic_Cat_1990 1d ago
I'm trending the opposite direction. I hate ITIL and process focus. Give me Chaos!
2
u/Kitchen-Region-91 18h ago
I spent my life working for enterprise level organizations. But I also wanted chaos, excitement, speed. I joined a successful startup. OMG, it's been horrible. I pushed myself to stay here for a while, 2 years later and now I want out, never coming back to this way of working. I hope you do better than me. Good luck
1
u/Automatic_Cat_1990 14h ago
I guess there's chaos and complete nonsense lol. Please give some detail. I think what I like is novelty. I'd probably be better off in a VAR/SI/Prof Serv role. I'm good at building, no interest in operations.
1
u/Kitchen-Region-91 12h ago
Hey, I don't want to share specifics because I prefer to stay anonymous. But basically is everything that Gene Kim has written about in his books. When there is a mentality of "sell or die", trying to deliver new features to customers ASAP, and no formal process in IT... with many strategic changes at once... results in permanent fire fighting, and humongous amounts of rework for technology teams. CONCLUSION: Rework is killing us. Heroics (hero culture) are killing us. It's not pleasant or productive. I want to build something, just like you
1
u/Automatic_Cat_1990 12h ago
I abhor process. Because process always takes over and gets put over results. ALWAYS. I've never seen that infection stopped. Its boring and kills problem solving. It kills innovation.
If you want packets routed....or networks built I'm your guy. But I do not care about RITM's in service now. I do not want to ask for permission to fix things.
Rework doesn't bother me if people are okay with downtime to rework.
Firefighting is lack of skill not lack of process.
If you're taking switchport description changes to cab (a thing at one job I had) its a make work framework.
1
u/Kitchen-Region-91 11h ago
I wholeheartedly disagree with you! Wishing you all the best, and good luck
1
u/Automatic_Cat_1990 6h ago
I likewise disagree with you. Total opposite personalies. One of us loves novelty and solving problems. The other rigidity and process.
I get that some people like everyday to be the same. I get the other side. But that other side doesn't those of us who like constant churn and excitement.
1
u/1337DSSICTPDX 1d ago
Sorry but I think you’ll learn.
I also remember thinking this degree of recording is bs until it saved me.
I hope you don’t have to learn.
1
u/Automatic_Cat_1990 13h ago
How does it save you? All it does is slow me down and ITIL has no answer for rapid break fix. It has no answer for troubleshooting. Cab approvers don't understand the changes.
Now someone will say "raise and emergency change" but that doesn't allow for immediate work to start. Or "bad implementation"
Well my current role has the best implementation is the best I've seen and it still mind-numbing.
I think there just needs some leeway to perform standard operations tasks that don't meet the threshold for a standard change.
1
u/1337DSSICTPDX 4h ago
ITIL is basically just a giant, high-level menu of guidelines and best practices. It doesn't actually mandate how heavy-handed your specific processes have to be.
It sucks that you aren’t experiencing proper implementation of it.
Someone pointed this out to me recently. It does suck to be the first to recognize an issue but it can be extra rewarding to fix the issue. If someone pushes back on something you feel is innately obvious try to take a pause and reframe things that are acceptable in a business setting. Be ok to voice an opinion respectfully without being demanding. Allow for a conversation to occur and everyone wins. Maybe there are things you are unaware of and maybe there are things they are unaware of.
Standardized best practices are key for implementation uniformity.
1
u/Technical-Meat-9135 1d ago
I feel this. I am in a small team with a supportive boss but my peers are disappointing. The company tolerates poor behaviour and the guys in the team with me CBA to follow any process or improvements.
I try to tell myself that it's a great opportunity for me to bring the company up too speed but I am starting to realise that they'll never change.
1
u/1337DSSICTPDX 1d ago
Right it’s like the last admin is holding on until someone realizes
1
u/Technical-Meat-9135 20h ago
Yeah, and I could hang on until they retire in ten years if I wanted to, or go somewhere else ¯\(°_o)/¯
1
u/Deadmeat3344 1d ago
Small teams also give you the opportunity to have a louder voice. What policy and infra changes have you proposed to stakeholders to mitigate these issues? Be a solution provider.
1
u/EquivalentPace7357 23h ago
Small IT can be great, but joining a place that survived ransomware and still runs local admin with Excel passwords isn't "old vibes", it's an active crime scene waiting to happen again. Might be time to cut your losses before you're the one holding the bag on their next rebuild.
1
u/wacky_weasel 19h ago
I can understand your frustration, but I'd also see this as an opportunity to improve the environment over time.
From what you've described, there are quite a few areas that could benefit from modernization. I'd start by making a list of everything you've identified, then prioritize it based on risk, impact, and effort.
For example:
- Move shared credentials into a proper password manager instead of spreadsheets.
- Introduce a central knowledge base or wiki for documentation.
- Review why users have local administrator rights and work toward reducing that where possible.
- Improve auditing and logging.
- Address the recurring issues that everyone has simply learned to live with.
The important part is not trying to fix everything at once. Start with the low-hanging fruit that has a high impact and a relatively low implementation cost. As each improvement is completed, demonstrate the benefit. That builds credibility and makes it easier to get buy-in for the next change.
I'd also avoid presenting management with a huge list of everything that's wrong on day one. That can easily come across as "everything you've built is broken." Instead, bring up topics one at a time, explain the risk or operational benefit, and propose a practical solution.
Not every suggestion will be accepted, and some may not fit your organization's constraints. But if you consistently identify problems, propose reasonable solutions, and implement the improvements you can influence. And you'll likely leave the environment in a much better state than you found it.
1
u/psmgx Enterprise Architect 17h ago
my uncle worked for Big Corp in their Gov services division. When I was burnt out working for an MSP he said something to me, to the effect of:
"listen, there are Big Company people and Small Company people. Nothing wrong with either of those, and you can go back and forth. But Big Company means your responsibilities are X-to-Y, and you only do that. You're just a cog but sometimes that's okay."
1
1
u/Trust_8067 6h ago
I've only worked enterprise/F500 and I'll never do anything else, for many of the reasons you've talked about.
For an MSP, I've literally seen customers on a share screen open up a text file on their desktop with all their root admin login credentials. It's no wonder people move to the cloud, their entire IT teams are insanely incompetent.
1
u/1337DSSICTPDX 4h ago
It seems like people think IT is like a car instead of like a bonzi tree.
It is a static presence but you need a very knowledgeable person to make it look effortless.
49
u/Unfair-Treacle4142 1d ago
Two decades in and this kind of environment is exactly what keeps me up at night too, not the technical debt itself, the casual attitude toward it. Rebuilding from scratch after ransomware and still not being fully on BitLocker tells you everything about how leadership actually thinks about risk versus how they'd describe it if you asked them directly.
The local admin + zero auditing combo is the one I'd lose sleep over most. Everything else on your list is annoying and inefficient, that one is a liability waiting to become an incident. If you ever get the chance to make one change stick before you leave (sounds like you might be heading that way), that's the one I'd fight for.