Feel free to edit this page to suggest tools to add, or make any other comments --Joey

lx

I have some code I wrote a decade ago which lists the 'extensions' in a folder. It's like ls -l (a little bit), but defaults to recursive, and colorized. For every extension type (defined as everything after the last dot, unless it starts with a dot) it lists the count and the the total size. If there's only one of that type then the output is a little different. I'd have to find the code and compile it, but it would be nice to release this in the wild.

datediff

I'd like to suggest a shell script I wrote long ago to manipulate dates and for which I still don't know any available replacement.

It computes the number of days between two dates and can check whether a date exists or not.

$ datediff 2009-09-27 2009-10-27
30
$ datediff 2009-09-27 2008-09-27
-365
$ datediff today
2455102
$ datediff today 0
2009-09-27
$ datediff test 2009-02-31
no
$ datediff test 2009-02-28
yes
$ datediff today -30
2009-08-28

vidir

A suggestion regarding vidir: I would find it quite useful if vidir could work with version control system commands like:

svn mv
svn rm

or

git mv
git rm

Another improvement would be the addition would be to add checkboxes to control which files should be controlled by the vcs and which shouldn't.

-- David Riebenbauer

What I want to do with vidir is make it able to run arbitrary configured commands based on filename transformations. So that it can be configured to run git add, or svn rm or whatever. Also so that if you remove .gz from a filename, it's decompressed; adding .gz compresses it, etc. I have not figured out a configuration language that is flexible enough to handle all these cases though. --Joey

How about adding copying ability to vidir? Items are recognized by the leading number in a line, right? A second instance of the same number should imply copying.

seddir?

Throwing out ideas for a vidir implementation to support non-interactive editors like sed.
An actual 'seddir' (what the name implies) wouldn't be too useful; the best option, in my opinion, would be an 'eddir' that supports both kinds of editors.

Firstly, a '--' option, which passes the rest of the arguments to the editor, would be nice. That would be enough to simulate a 'seddir' of some kind: VISUAL=sed vidir foo -- -i 's/foo/bar/'
Note the sad fact that we need to rely on the GNU extension '-i'. An option (say -s) to use stdin/stdout instead of creating a temp file, would be handy.
Also, talking about 'visual' is quite misleading here. Maybe an option (say -e) to manually specify the program to run, and fall back to the variables?

Currently printf 's/foo/bar/\nw\nq\n' | VISUAL=ed vidir works, but no way to combine that with vidir - (filelist on stdin).
An option (say -f) to specify a file that will be passed to the editor as stdin would be handy.

Summary:
'-f [script-file]' and '-s' are mutually exclusive.
'-f' is default, uses a temp file, passes contents of the file (script) specified in its optional argument as stdin to the editor. Can be repeated to supply more files on stdin.
'-s' uses stdin/stdout with the editor, instead of a temp file. (Not related to the stdin/stdout of eddir itself.)
'-e editor' specifies the program to run as the 'editor'. Defaults to ${VISUAL:-${EDITOR:-vi}} on '-f' and sed on '-s'.
Other arguments are file names as usual. (Also, IMO there's no need for '-' to enable the stdin filelist, just always use it if it's nonempty.)
'--' ends processing of arguments, and passes the rest to the editor.

What still cannot be done is to have a script (like for ed) on stdin. But how often do you have a script coming from a pipe anyway? Scripts are kept in files, so use '-f script'.

And here's some humour for you: Let's make a sedpe !!!

edlinks

Like vidir, to edit where symbolic links point at.

url2file

New tool url2file as found at http://specs.dachary.org/url2file/ : the idea is to be able to do wc -l $(url2file http://foo.com/)

See the dog utility from the package by that name.

ifne

I'm suggesting a command that would run the following command if and only if the standard input is not empty. I often want this in crontabs, as in:

find . -name core | ifne mail -s "Core files found" root

This is a good idea, and included now.

autopage

A command that pages the stdout of a subcommand only if stdout is a tty, similar to the way git treats its stdout. For example:

autopage ls -l

works like "ls -l | pager" but

autopage ls -l >mifilelist

works correctly and writes the output of "ls -l" in "myfilelist". This command would be useful for aliases, so you could add git's autopaging to other commands like this:

alias ls="autopage ls"

--Vicho

Might the generic unix tool that's missing here really be a command-line version of "isatty"? I like to think generic; running a pages if isatty is a specific case that can be done with a small shell script. --Joey

The generic unix tool is "test -t 1", so probably this shouldn't go on moreutils. Still I like the autopaging capability of git and I miss them in commands like grep or ls. On the other hand, I don't think every command should check if its stdout is a tty and run a pager if its output is larger than a screen. I think this should be done in a more general way, like globbing.--Vicho

moreutils tricks

Look at this poor man's hex editor, made with moreutils (and xxd)

xxd $file | vipe | xxd -r | sponge $file

For me 'vipe' is the killer app in moreutils, I hardly use the other (but I think it just takes getting used to the new repertoire). --ulrik.

That's sweet. I never thought of using vipe for that, which is of course, the point. Amusingly, for me, vipe is one of the rarer used tools. :-) --Joey

I'm sure you guys heard of mmv, and maybe zmv and others. Well, vidir seriously makes them look pathetic. Although it requires user interaction (and see "seddir" above, regarding that). --TaylanUB

URI to local path converter

This shell problem pops up in shell scripting for Nautilus and some other gnome applications. I'm proposing a tool to convert between file:/// URIs and local paths, with the proper encoding conversions in that of course.

Interface:

$ tourl "/tmp/my dir/idx.html"
file:///tmp/my%20dir/idx.html
$ tolocal "file://localhost/home/ben/Documents/report%202008.pdf"
/home/ben/Documents/report 2008.pdf

Just a suggestion, easily solved using most high-level desktop APIs. Is there already a shell tool? --ulrik

I imagine there's a perl oneliner that can do it. It doesn't seem generic enough for morutils. --Joey

body

If have a suggestion that may be a bit trivial, but it would be highly useful and make life a bit easier for the hacker: body. It belongs in a group with head and tail (hence the name). It outputs either one line from a file or a range of lines. Sure, you could say

head -n 42 file | tail -n 1

but isn't

body file:42

much more elegant? Note how you can just copy'n'paste your compiler error messages. Specifying ranges as

body file:40-44

will also save you some math that would slow down your workflow -- Jann

[Two weeks later:] This command can be emulated with sed:

sed -n 42p file
sed -n 40,44p file

Jann

about cattail

It can be done with "tail -fc+1 FOO".

This is missing the key part, which is exiting when the file stops being written to. --Joey

"null_tee" for sponge

I wrote a system that forks a lot of processes, redirecting stdout to YYYYMMDD-progname-stdout.txt and stderr to YYYYMMDD-progname-stderr.txt for each process. I also wrote a little utility I called 'null_tee' that would copy stdin to a file if there was data, else it would not create the file at all. With a spawn command of 'progname 2>&1 1>YYYYMMDD-progname-stdout.txt | null_tee -o YYYYMMDD-progname-stderr.txt', I would have all the stdout.txt files but only the stderr.txt files of programs that had actually sent stuff to stderr. In production mode, I redirected stdout to /dev/null so the only files that appeared were from unusual errors. It worked out quite well in practice.

It would be great if sponge had this same feature as an option: only create the file if something really came in from stdin. --KevinL

Use "foobar | ifne sponge file"? --TaylanUB

parallel

Need to resolve the name conflict with GNU parallel (http://www.gnu.org/software/parallel/). As far as I can tell, these are two distinct implementations addressing the same general problem space. --ckester

"sponge" being careful with symlinks

I wonder whether "sponge" will preserve symlinks and replace the content of a symlinked file with the new content (like expected from "editing in place" or from the usual shell redicrections to symlinks), as asked in a question at that Q&A site.

If yes, then it's one more advantage of "sponge" over "perl -i" (which is said not to follow symlinks according to its documentation) or "sed -i".

unfold

(Not sure whether I should add suggestions at the top or bottom of this page) We already have fold in standard distributions, which wraps lines to a specified length, but we don't have the reverse, to unwrap or unfold lines. I find this especially annoying when reading ebooks which have been pre-folded, as for instance Project Gutenberg texts are. If I try to read them on a narrower screen (e.g. my Palm computer) then it really screws up the text, so I put together this deceptively simple-looking one-liner:

sed ':a; /^$/!N; /\n$/!s/\n/\ /; ta' "$1" >"$1.unfolded"

It really needs some extra functionality though, for instance it should leave collections of short lines alone because they're probably verse (but how short?), and indented text currently gets big spaces in the middle of text when unfolded. Of course to make this more general-purpose the redirection to the '.unfolded' file should probably be removed.

This sed script is quite tricky and took me a lot of fiddling to come up with it. Here is a quick description:

Start with the label (:a) to mark the beginning of a loop. The semicolon separates sed commands. The first thing in the loop is a condition (/^$/!) testing if the current line is NOT blank then use the N command to append the next line to the current one. Next we have a condition (/\n$/!) that lets the rest of the loop work only if a newline is NOT at the end of line. If the appended line is empty the newline on the current line will be at the end of the line in the buffer, but if the appended line has text on it the newline will be wedged between text and won't be at the end, in which case the newline gets substituted with a space. And the 'ta' closes the loop.

haschanged

I don't think this is needed. Simply do this:

file_sig="`shasum $file`"
do_sth
[ "`shasum $file`" = "$file_sig" ] && do_sth_else