For a traditional software engineer? I retired last year after 3 decades and my salary was about the same as it was in the early 2000's at the last company I was at. Maybe I should have negotiated more but I thought only FAANG paid traditional pre-AI engineers more than $250K.
There is a tier just outside of FAANG that pays similarly or better, prominent examples being Uber, Airbnb, Stripe, Block, Databricks, Datadog, Pinterest, Snowflake, etc.
> fallback to running when the system comes online.
That isn't something I'd want to happen, it sounds like it creates a potential queue of scripts that will flood the system on start, if it works the way you described.
I prefer the deterministic behavior of cron, the script will run when it is specified to run, as you said earlier, as long as the system is running; and as I stated in a separate comment, it will run @reboot if I need it to run then.
> With @hourly I lose this control and multiple machines could potentially trigger backups at the same time
Then don't use @hourly, use staggered times, it's very easy.
> That isn't something I'd want to happen, it sounds like it creates a potential queue of scripts that will flood the system on start, if it works the way you described.
There are two options to fix it;
Disable persist so no catching up on missing scripts. Set OnBoot=5m so it gets ran 5 minutes after boot, so your script (say backup) is ran on boot first, then every time on schedule
Enable persist but just add sleep in ExecStartPre - very "cron" way but there is just no in-systemd option to enable "catch up" script to be delayed
Sadly no option to "run catch-up timers with delay" at least yet
> Then don't use @hourly, use staggered times, it's very easy.
Not in cron. In systemd it's just RandomizedOffsetSec=30m and it is "stable" - same host with same job will always have same delay so on multiple hosts it is spread nicely. There is also non-stable version
> That isn't something I'd want to happen, it sounds like it creates a potential queue of scripts that will flood the system on start, if it works the way you described.
This isn't what happens. If you leave it offline for days it'll only trigger the service only a single time.
I interpreted it more like "I have these 500 different cronjobs all spread out across $unit_of_time. If the system is down for longer than $unit_of_time and then comes back, does all 500 jobs start running instantly (since they missed their previous deadline)?"
Just to be clear, this isn't default systemd timer behaviour, you need to opt in by setting Persistent=true. If you have hundreds of jobs like this you need a proper queue and neither cronie nor systemd is the right tool because at that scale you'd surely need better observability
You could implement this with a gitlab instance in a separate system, like two VMs in proxmox or two physical machines, and a shell executor running in them. Gitlab CI has a nice feature to limit concurrency by using resource groups. Say you have 500 jobs spread through the day and the system stays offline for a while, when it comes online it'll start processing the jobs, but will only run a single one at a time. You get visibility, logs, queue monitoring and an API to query data.
> If you have hundreds of jobs like this you need a proper queue and neither cronie nor systemd is the right tool
Eh sometimes, but you can get pretty far with one of two approaches:
1. Careful use of Requires= and Wants= to group your scripts into chains of jobs, which achieves fixed parallel (though at 100s of jobs, I hope you're generating those unit files with a tool like Puppet or https://github.com/karlicoss/dron or something and not doing this by hand).
2. Even better, just use a lockfile. `ExecStart="flock -F $TMPDIR/mylock <command>"` is pretty hard to beat. Use -F so as not to confuse KillMode and resource accounting and you're golden. Just don't use flock(1) timeouts; let systemd handle that. Heck, if you have that many cron jobs, you should be doing this even if you don't use systemd; otherwise job latency changes can cause reboot-style thundering herds out of the blue.
1. This is spread across 500 files, maintainability goes out the window
2. If this for some reason fails, misconfiguration or unexpected shutdown, you could have a failure that's hard to track or debug
These are fine with a few services chained together, but this requires a shallow depth of dendencies. To have these theoretical hundreds of jobs chained together like this isn't practical or safe.
If you have 100 different jobs that were supposed to run over the past week, but didn't because offline, when you restart, they they all flood the system on start.
100 jobs all running at different times throughout the week is a very different load than them all falling back and running at the same time on system boot.
No, just different cron schedules. If I just reboot a machine the job doesn't get triggered, only if I start a machine after the cron schedule should have been triggered. To be fair, if I start two machines in these conditions this will happen too, but such situation is much more manageable than rebooting too machines in a short period of time.
I suggest trying out AMD. Not an evangelist by any means but on Linux there's really no choice. I have one myself since I moved from Nvidia and don't regret it.
> Outside of rare special cases, yes, they will still work using the Xwayland compatibility layer. It does a great job of providing compatibility for most X11 applications
Not on my 4 year old PC. Wayland performs poorly and usually in wonky ways. I tried it for several weeks. Could not stand the odd behaviors and poor performance and went back to X11. And this is an AlienWare PC, cost me $2000 US, the most I've ever paid for a PC. Can't imagine how bad Wayland would be on the lesser PC's/laptops in our home.
Xwayland != Wayland. Xwayland is an X server and a Wayland client. I have a feeling that whatever performed poorly in wonky ways was due to whatever you were attempting to run rather than Wayland itself.
If you see this comment and ever decide to give the wayland session and whatever you were trying to run another go, I'd be more than happy to try and help you fix it.
I'm not a Wayland evangelist but as someone with an old(custom built) desktop myself who has used the Wayland session since it became an option, your comment reads to me as:
"Xwayland(and by extension Wayland on KDE) is bad, I'm not going to list any specific programs I had issues with or any of the troubleshooting I tried in the several weeks I attempted to use it but for me it ran poorly and/or strangely.
I have an AlienWare PC, which should prevent any of these issues from happening(?)"
I have a 6 years old PC (around 1000€ at the time). The only problems I can think of were caused by Nvidia drivers. It's true there can be still some rough edges in Wayland, but at least for me, nothing that I can notice in my day to day.
> Can't imagine how bad Wayland would be on the lesser PC's/laptops in our home.
Thousands of users use daily distros based on Wayland without problems. Maybe you had bad luck and something on your system is not fully compatible. But for the majority of people Wayland works fine. Have you tried different distros?
Before I upgraded to my new PC (managed to get it just before AI sent prices into the stratosphere), I was using an older PC (2016~ish) without discrete GPU. It ran Plasma Wayland without any problems. My new PC (has an RX 7600) also runs it without any problems. I don't know what's wrong with your PC (I mean, AlienWare does make atrocious garbage, so it could be anything), but I expect it to run better on your lesser PC's/laptops.
They started doing that tracking where I worked until last summer, stress so high, GERD kicked in worse than it'd ever been and that's going back to the 1980's. I retired, and I do thank God I was able to. I now just dabble every week or so in my personal side projects.
For my selfhosting, I use local remotes the same as if I were using Github or Gitlab, as part of my CI tools chain, using a git hook script to kick off the Jenkins build on the remote directory. Everything is backed up daily and monthly (separately).
We use $200 inexpensive Lenovo laptops with 4 gigs of RAM and run KDE Plasma and Chrome as our streaming devices in the LR and MBR using air mouse device to control them. I also installed ZRAM on them. We could use Chromebooks but I like the idea of being in control of the OS in control of the machine.
For a traditional software engineer? I retired last year after 3 decades and my salary was about the same as it was in the early 2000's at the last company I was at. Maybe I should have negotiated more but I thought only FAANG paid traditional pre-AI engineers more than $250K.
reply