I’m curious how software can be created and evolve over time. I’m afraid that at some point, we’ll realize there are issues with the software we’re using that can only be remedied by massive changes or a complete rewrite.
Are there any instances of this happening? Where something is designed with a flaw that doesn’t get realized until much later, necessitating scrapping the whole thing and starting from scratch?
Happens all the time on Linux. The current instance would be the shift from X11 to Wayland.
And then ALSA to all those barely functional audio daemons to PulseAudio, and then again to PipeWire. That sure one took a few tries to figure out right.
And the strangest thing about that is that neither PulseAudio nor Pipewire are replacing anything. ALSA and PulseAudio are still there while I handle my audio through Pipewire.
How is PulseAudio still there? I mean, sure the protocol is still there, but it’s handled by
pipewire-pulseon most systems nowadays (KDE specifically requires PipeWire).Also, PulseAudio was never designed to replace ALSA, it’s sitting on top of ALSA to abstract some complexity from the programs, that would arise if they were to use ALSA directly.
Pulse itself is not there but its functionality is (and they even preserved its interface and pactl). PipeWire is a superset of audio features from Pulse and Jack combined with video.
For anyone wondering: Alsa does sound card detection and basic IO at the kernel level, Pulse takes ALSA devices and does audio mixing at the user/system level. Pipe does what Pulse does but more and even includes video devices
And then from ALSA to PulseAudio haha
They’re at different layers of the audio stack though so not really replacing.
Be careful what you wish for. I’ve been part of some rewrites that turned out worse than the original in every way. Not even code quality was improved.
In corporations, we call that job security.
Just rewriting the same thing in different ways for little gain except to say we did it
Funnily enough the current one is actually the one where we’ve made the biggest delta and it’s been worthwhile in every way. When I joined the oldest part of the platform was 90s .net and MSSQL. This summer we’re turning the last bits off.
Wayland, Pipewire, systemd, btrfs/zfs, just to name a few.
Wayland is THE replacement to broken, hack-driven, insecure and unmaintainable Xorg.
Pipewire is THE replacement to the messy and problematic audio stack on Linux to replace Pulseaudio, Alsa etc.
SystemD is THE replacement to SysVinit (and is an entire suite of software)
Like many, I am not a fan of SystemD and hope something better comes along.
Yes, I know. I was answering the question of if there were instances of this happening.
I would say the whole set of C based assumptions underlying most modern software, specifically errors being just an integer constant that is translated into a text so it has no details about the operation tried (who tried to do what to which object and why did that fail).
You have stderr to throw errors into. And the constants are just error codes, like HTTP error codes. Without it how computer would know if the program executed correctly.
You throw an exception like a gentleman. But C doesn’t support them. So you need to abuse the return type to also indicate “success” as well as a potential value the caller wanted.
So you need to abuse the return type to also indicate “success” as well as a potential value the caller wanted.
You don’t need to.
Returnung structs, returning by pointer, signals, error flags, setjmp/longjmp, using cxa for exceptions(lol, now THIS is real abuse).
Exceptionss are bad coding, and what’s abusive of using the full range of an integer? 0 success, everything else, error - check the API for details or call
strerror.Returning error codes in-band is the reason for a significant percentage of C bugs and security holes when the return value is used without checking. Something like Rust’s Result type that forces you to distinguish the two cases is much better design here. And no, you are not working with a whole language ecosystem of “sufficiently disciplined programmers” so that nobody ever forgets to check a return value.
Not to mention that errno is just a very broken design in the times of modern thread and event systems, signals, interrupts and all kinds of other ways to produce race conditions and overwrite the errno value before it is checked.
errno is bad programming.
stderr is useless if the syscall already returns a single integer only because of stupid C conventions.
Assembly doesn’t have concept of objects.
It does very much have the concept of objects as in subject, verb, object of operations implemented in assembly.
As in who (user foo) tried to do what (open/read/write/delete/…) to which object (e.g. which socket, which file, which Linux namespace, which memory mapping,…).
implemented in assembly.
Indeed. Assembly is(can be) used to implement them.
As in who (user foo) tried to do what (open/read/write/delete/…) to which object (e.g. which socket, which file, which Linux namespace, which memory mapping,…).
Kernel implements it in software(except memory mappings, it is implemented in MMU). There are no sockets, files and namespaces in ISA.
You were the one who brought up assembly.
And stop acting like you don’t know what I am talking about. Syscalls implement operations that are called by someone who has certain permissions and operate on various kinds of objects. Nobody who wants to debug why that call returned “Permission denied” or “File does not exist” without any detail cares that there is hardware several layers of abstraction deeper down that doesn’t know anything about those concepts. Nothing in the hardware forces people to make APIs with bad error reporting.
And why “Permission denied” is bad reporting?
Because if a program dies and just prints strerror(errno) it just gives me “Permission denied” without any detail on which operation had permissions denied to do what. So basically I have not enough information to fix the issue or in many cases even to reproduce it.
It may just not print anything at all. This is logging issue, not “C based assumption”. I wouldn’t be surprised if you will call “403 Forbidden” a “C based assumtion” too.
But since we are talking about local program, competent sysadmin can
straceprogram. It will print arguments and error codes.
You mean 0 indicating success and any other value indicating some arbitrary meaning? I don’t see any problem with that.
Passing around extra error handling info for the worst case isn’t free, and the worst case doesn’t happen 99.999% of the time. No reason to spend extra cycles and memory hurting performance just to make debugging easier. That’s what debug/instrumented builds are for.
Passing around extra error handling info for the worst case isn’t free, and the worst case doesn’t happen 99.999% of the time.
The case “I want to know why this error happened” is basically 100% of the time when an error actually happens.
And the case of “Permission denied” or similar useless nonsense without any details costing me hours of my life in debugging time that wouldn’t be necessary if it just told me permission for who to do what to which object happens quite regularly.
“0.001% of the time, I wanna know every time 👉😎👉”
Yeah, I get that. But are we talking about during development (which is why we’re choosing between C and something else)? In that case, you should be running instrumented builds, or with debug functionality enabled. I agree that most programs just fail and don’t tell you how to go about enabling debug info or anything, and that could be improved.
For the “Permission Denied” example, I also assume we’re making system calls and having them fail? In that case it seems straight forward: the user you’re running as can’t access the resource you were actively trying to access. But if we’re talking about some random log file just saying “Error: permission denied” and leaving you nothing to go on, that’s on the program dumping the error to produce more useful information.
In general, you often don’t want to leak more info than just Worked or Didn’t Work for security reasons. Or a mix of security/performance reasons (possible DOS attacks).
During development is just about the only time when that doesn’t matter because you have direct access to the source code to figure out which function failed exactly. As a sysadmin I don’t have the luxury of reproducing every issue with a debug build with some debugger running and/or print statements added to figure out where exactly that value originally came from. I really need to know why it failed the first time around.
As sysadmin you should know about strace
I know about strace, strace still requires me to reproduce the issue and then to look at backtraces if nobody bothered to include any detail in the error.
Somehow (lack of) backtrace and details in error is “C based assumption”
Yeah, so it sounds like your complaint is actually with application not propagating relevant error handling information to where it’s most convenient for you to read it. Linux is not at fault in your example, because as you said, it returns all the information needed to fix the issue to the one who developed the code, and then they just dropped the ball.
Maybe there’s a flag you can set to dump those kinds of errors to a log? But even then, some apps use the fail case as part of normal operation (try to open a file, if we can’t, do this other thing). You wouldn’t actually want to know about every single failure, just the ones that the application considers fatal.
As long as you’re running on a turing complete machine, it’s on the app itself to sufficiently document what qualifies as an error and why it happened.
Ugh, I do not miss C…
Errors and return values are, and should be, different things. Almost every other language figured this out and handles it better than C.
Errors and return values are, and should be, different things.
That’s why errno and return value are different things.
It’s more of an ABI thing though, C just doesn’t have error handling.
And if you do exception handling wrong in most other languages, you hamstring your performance.
The unofficial C motto “Make it fast, who gives a shit about correctness”
Alsa > Pulseaudio > Pipewire
About 20 xdg-open alternatives (which is, btw, just a wrapper around gnome-open, exo-open, etc.)
My session scripts after a deep dive: full rewrite, structured to be Xorg/waylamd/terminal independent, uses sx instead startx and custom startxfce4 as base. Seriously, startxfce4 has workarounds from the 80ies and software rot affected formatting already.
Turnstile instead elogind (which is bound to systemd releases)
mingetty, because who uses a modem nowadays?
About 20 xdg-open alternatives (which is, btw, just a wrapper around gnome-open, exo-open, etc.)
I use handlr-regex, is it bad? It was the only thing I found that I could use to open certain links on certain web applications (like android does), using exo-open all links just opened on the web browser instead.
ALSA is based
Pulseaudio doesn’t replace ALSA. Pulseaudio replaces esd and aRts
those last two are just made up words
All words are made up
Linux could use a rewrite of all things related to audio from kernel to x / Wayland based audio apps.
Wayland could already do with a replacement.
Yup, Wayland is so old it already has old concepts. But it is also changing a lot
Needs to be replaced already. They’re having to change to explicit sync, which they should have done from the start. So throw it out, start over, make X12.
Wayland is incomplete and unfinished, not broken and obsolete and hopelessly bad design. PulseAudio was bad design. Wayland is very well designed, just, most things haven’t been ported for it yet and some design by committee hell, but even that one is kind of a necessary tradeoff so that Wayland actually lasts a long time.
What people see: lol Firefox can’t even restore its windows to the right monitors
What the Wayland devs see: so how can we make it so Firefox will also restore its windows correctly on a possible future VR headset environment where the windows maintain their XYZ and rotation placement correctly so the YouTube window you left above the stove goes back above the stove.
The Wayland migration is painful because they took the occasion to redo everything from scratch without the baggage of what traditional X11 apps could do, so there is less likely a need for a Wayland successor when new display tech arrives and also not a single display server that’s so big its quirks are now features developers relied on for 20 years and essentially part of the standard.
There’s nothing so far that can’t be done in Wayland for technical implementation reasons. It’s all because some of the protocols aren’t ready yet, or not implemented yet.
Can’t even update Firefox in place. Have to download a new copy, run it from the downloads folder, make a desktop shortcut myself, which doesn’t have the Firefox icon.
Can’t remember if that was mint or Ubuntu I was fiddling with, but it’s not exactly user friendly.
Do not download Firefox of the internet. Use your package manager or flatpak
This has nothing to do with Wayland, it’s just AppImages kinda sucking. Use Flatpak or the one in your distro’s repos, not the AppImage. AppImages are the equivalent of portable apps on Windows, like the single exe ones you’d put on a flash drive to carry around.
Also the AppImage developer is very against Wayland and refuses to support it, which is why Wayland support is a shitshow on AppImages.
If you pick the Flatpak it’ll get updated in the background, have a proper launcher and everything.
X11 is 40 years old. I’d say it’s been rather successful in the “won’t need to be replaced for some time” category. Some credit where due.
There’s nothing so far that can’t be done in Wayland for technical implementation reasons. It’s all because some of the protocols aren’t ready yet, or not implemented yet.
I mean … It doesn’t matter why it can’t be done. Just that it can’t be done.
40 years old is also what makes it so hard to replace or even reimplement. The bugs are all decade old features, everything is written specifically for Xorg, all of which needs to be emulated correctly. It sure did serve us well, it’s impressive how long we’ve managed to make it work with technology well beyond the imagination of the engineers in the 80s.
There’s this for the protocols: https://github.com/probonopd/wayland-x11-compat-protocols
It can be done, it’s just nobody wants to do it. It’s not really worth the effort, when you can work on making it work properly in Wayland instead. That way you don’t need XWayland in the first place, but also XWayland can then implement it using the same public API everyone else does so it works on every compositor.
There’s nothing so far that can’t be done in Wayland for technical implementation reasons.
Then make it fully X11 backwards compatible. Make Wayland X12. C’mon, they already admitted NVidia was right and are switching the sync and working to finally support the card they’ve been busting a hate boner over the driver simply because they’re bigots against the licensing. Time to admit breaking the world was a mistake, too.
I can’t up-vote this enough. The “architectural purists” have made the migration a nightmare. Always blaming everyone else for simply not seeing their genius. I’m honestly surprised it’s gotten as far as it has.
It’s slowly happening. KDE can now do global Xwayland shortcuts, they also implemented XWaylandVideoBridge and compositor restart crash recovery for apps. We’re getting proper HDR, we have proper per-monitor refresh rates and VRR, I can even hotplug GPUs. Some of that stuff works better in XWayland because we can just run multiple instances with different settings. For the particularly stubborn cases, there’s rootful XWayland. X12 would have to break things too, and I doubt an Xorg rewrite would be all that much further than Wayland is. Canonical had a go at it too with Mir which was much less ambitious.
NVIDIA was right on that one indeed, but Wayland also predates Vulkan and was designed for GLES, pretty much at the tail end of big drivers and the beginning of explicit and low level APIs like Vulkan. They could very well have been right with EGLStream too, but graphics on Linux back then was, erm, bad. But in the end they’re all still better than the kludge that is 3D in Xorg.
It’s getting a lot of momentum and a lot of things are getting fixed lately. It went from unusable to “I can’t believe it’s not Xorg!” just this year for me. It’s very nice when it works well. We’ll get there.
Agreed, Wayland has a monumental task to do: replacing a 30+ year old windowing system.
Seriously, I’m not a heavy software developer that partakes in projects of that scale nor complexity but just seeing it from the outside makes me hurt. All these protocols left-right and center, surely just an actual program would be cleaner? Like they just rewrite X from scratch implementing and supporting all modern technology and using a monolithic model.
Then small projects could still survive since making a compositor would almost be trivial, no need to rewrite Wayland from scratch cause we got “Waykit” (fictional name I just thought of for this X rewrite), just import that into your project and use the API.
Wayland and X are very very different. The X protocol is a protocol that was designed for computer terminals that connected into a mainframe. It was never designed for advanced graphics and the result is that we have just built up a entire system that balances on a shoe box.
Wayland is a protocol that allows your desktop to talk to the display without a heavy server. The result is better battery life, simplified inputs, lower latency, better performance and so on
That would work if the only problem they wanted to solve was an outdated tech stack for X. But there are other problems that wayland addresses too, like: how to scale multiple monitors nicely, is it a good idea to give all other apps the keystrokes that you do in the one in focus (and probably a lot more)
I agree in the sense that Wayland adoption would have definitely gone quicker if that was the case, however in the long run this approach does make sense (otherwise you will eventually just run into the same sorts of issues X11 had).
Btw what you’re describing is not that far off from the normal way of using Wayland protocols in development - you use wayland-scanner to generate C source files from the protocols, and you include those to actually “use” the protocols in your programs. Admittedly all my Wayland development experience has been “client-side”, so I really don’t know how complex it is to build a compositor, but dwl (minimalist Wayland compositor) is only around 3k lines of code (only slightly more than dwm (minimalist X wm)).
It’s what happens when you put theory over practicality.
What we wanted: Wayland.
What we needed: X12, X13…
What was stopping X just undergoing some gutting? I get it’s old and covered in dust and cobwebs but look, those can be cleaned off.
“Scoop out the tumors, and put some science stuff in ya”, the company that produced that quote went on to develop the most advanced AGI in the world and macro-scale portable on-demand indestructible teleportation.
Because we no longer have mainframes in computer labs. Each person now has there own machine.
And yet I play modern games on modern hardware with X just fine. It’s been extended a little bit since the 80s.
Yes it works but it everything is glued together with duct tape
I would rather X didn’t get access to deadly neurotoxin, thanks
I dunno, sounds kinda cool.
X12 it’s got 15% less X11!
The X standard is a really big mess
That’s kind of what I was trying to imply.
We needed a new X with some of the archaic crap removed. I.e. no one needs X primitives anymore, everything is its own raster now (or whatever it’s called).
Evolving X would have given us incremental improvements over time… Eventually resulting in something like Wayland.
You can’t evolve something that old.
No body wanted Wayland except the mad scientists and anti nvidia bigots that made it.
Imagine calling developers who have a cold relationship with Nvidia due to Nvidia doing the bare minimum for Linux development “bigots” lol
I think you must be a fanboy.
I’m no fanboy of any video card. I just have ton of laptops with NVidia in them, and the bigots making Wayland never gave a darn about our plight… and then they started pushing distros to switch before they did anything to fix it. Their callous attitude toward the largest desktop linux userbase is insulting and pushing the distros before they fix the problem should be criminal. Every one of them should be put away for trying to ruin Linux by abandoning it’s largest desktop user base. We dislike them, dislike them so much.
Now, will it keep us from using that crap when it finally works? No. We don’t have much choice. They’ve seen to that. x11 will go the way of the dodo. But can we dislike them forever for dragging us through the mud until they were finally forced to fix the darn thing? Yeah. Wish them nothing but the worst.
It is so much better than X
Some form of stable, modernized bluetooth stack would be nice. Every other bluetooth update breaks at least one of my devices.
I realize that’s not exactly what you asked for but Pipewire had been incredibly stable for me. Difference between the absolute nightmare of using BT devices with alsa and super smooth experience in pipewire is night and day.
upstart, unity, mir, snap, ubuntu
Not too relevant for desktop users but NFS.
No way people are actually setting it up with Kerberos Auth
100% this
We need a networked file system with real authentication and network encryption that’s trivial to set up and that is performant and that preserves unix-ness of the filesystem, meaning nothing weird like smb, so you can just use it as you would a local filesystem.
The OpenSSH of network filesystems basically.
So sshfs or sftp?
Performance of those is atrocious.
lol that someone already said Wayland.
Your welcome. :)
His welcome?
That’ll teach me to type when I’m mad. “You’re”. There a go.
Actually, it won’t teach me. Wayland’s mere existence will bother me till the day I die, I’m sure. Especially once it’s working well enough for me to have to adopt it. The resentment will grow.
There’s already a lot of people rewriting stuff in Rust and Zig.
What are the advantages of Zig? I’ve seen lots of people talking about it, but I’m not sure I understand what it supposedly does better.
Tiny learning curve, easy to refactor existing projects
The goal of the zig language is to allow people to write optimal software in a simple and explicit language.
It’s advantage over c is that they improved some features to make things easier to read and write. For example, arrays have a length and don’t decay to pointers, defer, no preprocessor macros, no makefile, first class testing support, first class error handling, type inference, large standard library. I have found zig far easier to learn than c, (dispite the fact that zig is still evolving and there are less learning resources than c)
It’s advantage over rust is that it’s simpler. Ive never played around with rust, but people have said that the language is more complex than zig. Here’s an article the zig people wrote about this: https://ziglang.org/learn/why_zig_rust_d_cpp/
dmesg
/jk
Linux does this all the time.
ALSA -> Pulse -> Pipewire
Xorg -> Wayland
GNOME 2 -> GNOME 3
Every window manager, compositor, and DE
GIMP 2 -> GIMP 3
SysV init -> SystemD
OpenSSL -> BoringSSL
Twenty different kinds of package manager
Many shifts in popular software
BoringSSL is not a drop-in replacement for openssl though:
BoringSSL is a fork of OpenSSL that is designed to meet Google’s needs.
Although BoringSSL is an open source project, it is not intended for general use, as OpenSSL is. We don’t recommend that third parties depend upon it. Doing so is likely to be frustrating because there are no guarantees of API or ABI stability.
Omg nobody has mentioned FHS?!
What that?
Amended my post 😉
Starting anything from scratch is a huge risk these days. At best you’ll have something like the python 2 -> 3 rewrite (leaving scraps of legacy code all over the place), at worst you’ll have something like gnome/kde (where the community schisms rather than adopting a new standard). I would say that most of the time, there are only two ways to get a new standard to reach mass adoption.
-
Retrofit everything. Extend old APIs where possible. Build your new layer on top of https, or javascript, or ascii, or something else that already has widespread adoption. Make a clear upgrade path for old users, but maintain compatibility for as long as possible.
-
Buy 99% of the market and declare yourself king (cough cough chromium).
Python 3 wasn’t a rewrite, it just broke compatibility with Python 2.
In a good way. Using a non-verified bytes type for strings was such a giant source of bugs. Text is complicated and pretending it isn’t won’t get you far.
-
