Introducing Paled - Motivations
There have been many shells on Unix/Linux: Thompson, Mashey, Bourne, C, Korn, Bash, Zsh, Fish, and more. However, many of them are mostly the same due to compatibility problems, Csh and Fish having the most different shell syntax. Paled is yet another shell, but with different goals...
I got the main idea for this project in 2019, after reading a book called The UNIX-Haters Handbook. This book provided valuable opinions on the problems the majority of Linux shells have still to this day. With Paled, I set out to fix some of these problems.
The Problems
There are a few problems with current shells. Windows' Cmd isn't powerful enough, Powershell is too OOP-like, and Sh, Bash, and the like are too inconsistent, too outdated, sometimes too hard to use or unintuitive, and too similar (due to POSIX standards) which prevents them from trying to fix many of these problems with shells, as does needing to be backwards compatible.
Unix shells have many problems. Their commands and flags are often very cryptic and hard to remember at times. This was done at a time when you needed to save as much typing (and printing) as possible due to being on slower machines and display types (like teletypes or terminals). We no longer have this restriction, especially when we can now utilize helpful features like autocomplete. Unix terminal tools, and the shell itself, also reduces error reporting to cryptic, or sometimes inexistant, messages for this same reason.
Unix shells also have problems with file globbing. Because of Unix and Linux's lax rules towards file names, file globbing can cause problems when a file is named similarly to a flag. Additionally, there's no way to distinguish between two separate patterns when doing file globbing. For example, this is impossible in unix shells:
Because of Unix and Linux's care-free attitude towards file deletions, accidentally deleted files are irrecoverable, or rather, not easy to recover. This is where the shell can help out, by having commands that trash things instead of deleting them outright. Of course, unix shells don't do this.
A lot of the tools used in Unix/Linux shells rely on the idea of pipes. The problem is pipes are not intuitive at all. Additionally, many of these Unix/Linux shells have been around for so long that they've gathered a bunch of bloat and inconsistency when it comes to cli flags/options. The "ls" command has over 40 flags, for example.
The Unix philosophy was to make programs that do only one thing and do it well, and to use pipes to compose programs together. However, this has been lost. "ls" does things that have historically been done with other tools. "find" does things "cpio" can do in addition to actually finding files. Some commands have similar features but with completely different (inconsistent) flags to use those features. Furthermore, some commands have almost the exact same functionality, but have been completely separated out into different commands.
Additionally, Unix/Linux Shell tools are very inconsistent when it comes to how flags/options are typed. Some programs use "-" for single letter options, some use them for multi-letter options. Some use "--" only for multi-letter options, some use them for single-letter options also. Some use "=[value]", while others put the value immediately after the option flag. Furthermore, flags on Windows are prefixed with "/".
Shell syntax can be ugly, inconsistent, and tedious, especially when writing scripts. Shells have the ability to run scripts. However, this is actually a further restriction for users, because now not only do you have to make sure you have the right CPU for binaries as well as the right OS, you also have to make sure you have the right script for your shell, where that script *also* has to work for your OS (since there's different tools on different OS's, and different ways to do things). And furthermore, Shell scripts are not actually that powerful anyways! In order to run a script for a certain shell, either your shell has to support that script type, or you have to install a shell - which includes both the cli interface as well as running scripts - to run that script. Meanwhile, we have things like Python, Lua, etc. which can actually be more portable and are designed solely for scripts, and are far more flexible than shell scripts.
This is why I'm hesitant to include scripts into Paled. They don't seem to be needed for a shell. When someone needs to script, they can use something much better for the job with less hassle. Paled's first focus should be its interactive use as a cli. I might consider other options for scripts or batching later on.
The Fixes
Paled is a complete do-over. It gets rid of trying to be compatible with anything else in hopes that it might fix all of these problems. The very first problem to be fixed is syntax. We need the ability to handle arrays (of files, etc. - or, patterns) for one specific argument of a program. We need a better help system. We need better autocomplete for *all* programs. We need to get rid of flags and have named arguments that can have defaults. We need a way to handle subcommands. We need type-checking. We need better error handling. We also need to find ways to make pipes easier to use.
Paled also has the chance to bring back some features from older shells, namely TOPS-20, that have been lost over the years. This includes batching of subcommands, as well as a consistent subcommand cli, and TOPS-20's Load-class commands to help with development.
Paled also has the chance to fix many of the problems and bloat of things like GNU's tools. Paled isn't just a shell. It also includes the most-needed tools to do work within the shell. This should include deleting/trashing, moving, listing, printing files, finding files, searching within files, etc.
Finally, Paled should be cross-platform.
Paled's Goals
Current Status
Currently, Paled and its tools are written in C. The tools include a basic command line interface, some builtins for changing directories, for getting environment variables, and for printing the directory history, tools for listing files in directories, printing files, moving files/directories, renaming and copying files/directories, and removing files/directories. Paled also has Path Hashing and partial support for Command history.
Paled also currently only has Linux support.
The plans are to provide more tools, add support for Windows, Mac, and FreeBSD, Autocompletion, a trash tool, aliases, job control, as well as a rewrite to Odin.
The Odin rewrite is coming along well. I have implemented path hashing, parsing, and calling out to external programs.
I got the main idea for this project in 2019, after reading a book called The UNIX-Haters Handbook. This book provided valuable opinions on the problems the majority of Linux shells have still to this day. With Paled, I set out to fix some of these problems.
The Problems
There are a few problems with current shells. Windows' Cmd isn't powerful enough, Powershell is too OOP-like, and Sh, Bash, and the like are too inconsistent, too outdated, sometimes too hard to use or unintuitive, and too similar (due to POSIX standards) which prevents them from trying to fix many of these problems with shells, as does needing to be backwards compatible.
Unix shells have many problems. Their commands and flags are often very cryptic and hard to remember at times. This was done at a time when you needed to save as much typing (and printing) as possible due to being on slower machines and display types (like teletypes or terminals). We no longer have this restriction, especially when we can now utilize helpful features like autocomplete. Unix terminal tools, and the shell itself, also reduces error reporting to cryptic, or sometimes inexistant, messages for this same reason.
Unix shells also have problems with file globbing. Because of Unix and Linux's lax rules towards file names, file globbing can cause problems when a file is named similarly to a flag. Additionally, there's no way to distinguish between two separate patterns when doing file globbing. For example, this is impossible in unix shells:
1 | mv *.md ../*.txt |
Because of Unix and Linux's care-free attitude towards file deletions, accidentally deleted files are irrecoverable, or rather, not easy to recover. This is where the shell can help out, by having commands that trash things instead of deleting them outright. Of course, unix shells don't do this.
A lot of the tools used in Unix/Linux shells rely on the idea of pipes. The problem is pipes are not intuitive at all. Additionally, many of these Unix/Linux shells have been around for so long that they've gathered a bunch of bloat and inconsistency when it comes to cli flags/options. The "ls" command has over 40 flags, for example.
The Unix philosophy was to make programs that do only one thing and do it well, and to use pipes to compose programs together. However, this has been lost. "ls" does things that have historically been done with other tools. "find" does things "cpio" can do in addition to actually finding files. Some commands have similar features but with completely different (inconsistent) flags to use those features. Furthermore, some commands have almost the exact same functionality, but have been completely separated out into different commands.
Additionally, Unix/Linux Shell tools are very inconsistent when it comes to how flags/options are typed. Some programs use "-" for single letter options, some use them for multi-letter options. Some use "--" only for multi-letter options, some use them for single-letter options also. Some use "=[value]", while others put the value immediately after the option flag. Furthermore, flags on Windows are prefixed with "/".
Shell syntax can be ugly, inconsistent, and tedious, especially when writing scripts. Shells have the ability to run scripts. However, this is actually a further restriction for users, because now not only do you have to make sure you have the right CPU for binaries as well as the right OS, you also have to make sure you have the right script for your shell, where that script *also* has to work for your OS (since there's different tools on different OS's, and different ways to do things). And furthermore, Shell scripts are not actually that powerful anyways! In order to run a script for a certain shell, either your shell has to support that script type, or you have to install a shell - which includes both the cli interface as well as running scripts - to run that script. Meanwhile, we have things like Python, Lua, etc. which can actually be more portable and are designed solely for scripts, and are far more flexible than shell scripts.
This is why I'm hesitant to include scripts into Paled. They don't seem to be needed for a shell. When someone needs to script, they can use something much better for the job with less hassle. Paled's first focus should be its interactive use as a cli. I might consider other options for scripts or batching later on.
The Fixes
Paled is a complete do-over. It gets rid of trying to be compatible with anything else in hopes that it might fix all of these problems. The very first problem to be fixed is syntax. We need the ability to handle arrays (of files, etc. - or, patterns) for one specific argument of a program. We need a better help system. We need better autocomplete for *all* programs. We need to get rid of flags and have named arguments that can have defaults. We need a way to handle subcommands. We need type-checking. We need better error handling. We also need to find ways to make pipes easier to use.
Paled also has the chance to bring back some features from older shells, namely TOPS-20, that have been lost over the years. This includes batching of subcommands, as well as a consistent subcommand cli, and TOPS-20's Load-class commands to help with development.
Paled also has the chance to fix many of the problems and bloat of things like GNU's tools. Paled isn't just a shell. It also includes the most-needed tools to do work within the shell. This should include deleting/trashing, moving, listing, printing files, finding files, searching within files, etc.
Finally, Paled should be cross-platform.
Paled's Goals
- Designed primarily for interactive use
- Type checking
- Arrays that can be passed into programs
- Better file globbing/patterns
- Arguments, named arguments, optional arguments, default argument values
- Subcommands
- Better errors
- Easy to use and consistent set of tools
- Cross-platform - Windows, Mac, Unix/Linux, Haiku, FreeBSD
Current Status
Currently, Paled and its tools are written in C. The tools include a basic command line interface, some builtins for changing directories, for getting environment variables, and for printing the directory history, tools for listing files in directories, printing files, moving files/directories, renaming and copying files/directories, and removing files/directories. Paled also has Path Hashing and partial support for Command history.
Paled also currently only has Linux support.
The plans are to provide more tools, add support for Windows, Mac, and FreeBSD, Autocompletion, a trash tool, aliases, job control, as well as a rewrite to Odin.
The Odin rewrite is coming along well. I have implemented path hashing, parsing, and calling out to external programs.
Seems interesting.
When you speak about auto-completion, do you mean you can suggest sub commands and parameters ? How do you plan to make that work with existing programs ?
When you speak about auto-completion, do you mean you can suggest sub commands and parameters ? How do you plan to make that work with existing programs ?
On discord he mentioned that he'll be reimplementing existing programs to work with shell - so no support for existing programs.
Nice, looking forward to using it. Re above: Does that mean your're implementing all the processes yourself instead of running a terminal behind the scenes like com-emu does?
The suggestions here about future Terminals sound really nice (Bitmap rendering, do more with mouse pointer and scripting language)
The suggestions here about future Terminals sound really nice (Bitmap rendering, do more with mouse pointer and scripting language)
mmozeiko
On discord he mentioned that he'll be reimplementing existing programs to work with shell - so no support for existing programs.
Appreciate that update Mārtiņš, as I was wondering about this myself (and I don't use Discord anymore!)
mrmixer
When you speak about auto-completion, do you mean you can suggest sub commands and parameters ? How do you plan to make that work with existing programs ?
So, about the auto-completion. I haven't figured this out quite yet. Bash does programmable auto-completion by letting you write a script that can do this. I might go with something similar - an extra file - but, it won't be a shell-script file. I'll have to think about that more.
OliverMarsh
Does that mean your're implementing all the processes yourself instead of running a terminal behind the scenes like com-emu does?
So, this actually isn't a terminal emulator. It's a shell. I'm considering writing another article to cover this. Shells are the programs that start up other programs. This includes graphical shells as well (ex: explorer.exe). If the shell is a cli, it communicates with the terminal emulator to print the prompt, just like every other application that prints to the terminal emulator. Technically, you can also have a TUI shell too, which might be something interesting to experiment with.
mmozeiko
On discord he mentioned that he'll be reimplementing existing programs to work with shell - so no support for existing programs.
abnercoimbre
Appreciate that update Mārtiņš, as I was wondering about this myself (and I don't use Discord anymore!)
Sorry I haven't said anything for a while. I still need to think about many things for this project, since I am doing things very differently from existing shells.
So, the plan is that I am going to reimplement many GNU tools, but won't be restricting myself to the interface they provide.
However, this does pose problems with interacting with all of the existing tools. There's no possible way I can reimplement all of them of course. Ways to handle this could be:
1.) Add support for Paled as a PR to the upstream repository
2.) Create wrapper programs - a program that will take arguments in Paled-style and exec the existing tool and hand them over in the way they expect
3.) Create some sort of file that tells Paled how to do the conversion from Paled-style arguments to .... C-style/normal-style.
There's other problems like, if I wanted return values from programs to be more useful in Paled, how would I do that so existing programs and OS apis work with that. Linux/Unix (and perhaps Windows too) return an integer on process completion. I believe Plan9 returns a string.
Perhaps I can solve this by creating an additional pipe, or just using the stdout or stderr pipe (depending on how I want return values to work).
I am planning on writing more articles to talk about all of these things. I'm just a slow writer :)
That's quite funny, I was toying with making a filemanager a few years ago and got to think of it as a shell, and started giving him a "console" like interface. So I kinda drew the same conclusions from the opposite path ^^
I can't wait to use Paled :)
I can't wait to use Paled :)