Why are you using meow? It seems like you are just using it to grab args and print help. Does it do something that is not immediately apparent? It has a lot of dependencies that might be completely useless for you.
log-symbols seems also completely useless dependency, you could just add the two symbols you are using by hand. Is it doing something special?
As for nodejs making things easier. Maybe it's just what you are used to. There isn't much code here, but I'm guessing node_modules is going to be pretty large. I think you could make this work just as well using any language. I guess the bonus here is that this is portable (compared to implementing with bash), but since youtube-dl (as far as I know) is made with Python, maybe it would have been better to make this TUI using Python as well since at least the user will have it installed.
Thanks for making this. I'd say that Node does not make good CLI apps though. It means I have to install it on my machine in order to use your program. The best CLI apps are made with compiled languages so you can distributed them without a runtime, third party libs...
Just so you know, the tool is great, the code looks fine and the link is appreciated. I think the reason why you get down voted is because your comment could be read as you humble bragging.
Don't take it personally though, I've done similar posts/comments in the past and it took me quite some time to realize comments with this tone can be discouraging for people just getting into software engineering.
So please continue creating awesome tools like this and share it with us all and don't be discouraged. We all love to see tools like this. :)
That's great, it's a cool tool, but I don't understand why this couldn't have been done in a similar amount of time in a different language. I think if you replace "nodejs" with "English" in the sentence it might explain why people are downvoting; it just doesn't really mean anything.
I recently wrote[1] a track-selector front end script for youtube-dl that uses yad[2] as a simple GUI[3]. It's not finished (it needs better error handling anbd the gui layout is kind of ugly), but it mostly works.
It automatically places videos in the configured folder based on type. Just type in an artist name, and select the type of video. Saves me a ton of time.
It's easy to extend and I plan to generalize it and translate it to bash sometime soon. I just do all of my media consumption in Windows currently.
Cool, here's my simple script to select format called vdl, and can be extended:
#!/bin/bash
OPTS="-o %(title)s.%(ext)s --console-title --restrict-filenames"
if [ "$#" -eq 2 ]; then
if [ "$2" -eq "251" ]; then
OPTS="$OPTS -x"
fi
echo youtube-dl $OPTS -f $2 $1
youtube-dl $OPTS -f $2 $1
else
echo youtube-dl -F $*
youtube-dl -F $*
echo
echo -n "Enter number: "
read fmt
youtube-dl $OPTS -f $fmt $*
fi
To explain, if you know the format number off the top of your head you can pass it after the video id, otherwise it will prompt. Might upgrade it one day to use "set -x" instead of echo and exec.
However, the youtube (and other sites id) and the format number will never contain spaces, so quoting isn't really needed here. If it gives peace of mind, sure.
I'm generally against adding interactive features to CLI programs if there isn't a good reason for that. I don't want to remember which options I can/can't use in a non-interactive script.
Plus youtube-dl already has multiple GUI/wrappers, so there's already a separation for interactive features.
Don’t see why, if anything more people should know about it so shared content can become more redundant as people download it across the world. YouTube is a fairly brittle hub sometimes with all the takedowns.
I suspect the GP is concerned Youtube will take steps to block it.
This scares me a bit too, I watch basically all Youtube content via Youtube-dl. I have a script that runs every night on Mac, so that new content automagically appears in my Movies folder.
Personally, I subscribe to YouTube channels via RSS (a little known but existent feature). Then I download all the videos I want to watch with `youtube-dl -a -`. No buffering, any video window size I want, finer-grained speed playback in VLC, and no Google account required.
Launchd + bash script. The channels and playlists to download from are managed in a text file. I really need to get around to throwing this up on Github, I’ll see if I can find some time to do that in the next few days and let you know.
If Youtube ever decides to employ encrypted media extensions or similar, youtube-dl will effectively cease as a useful tool. The analog loophole will always exist but that’s not something I want to use day-to-day.
I have been a user for many years and it feels like the distro package works at least 51/52 weeks a year, so I'm not sure it's an issue more than with any other package.
youtube-dl has an update command built in that bypasses the distribution's update system. Works even on Debian stable.
You could argue this is bad because it bypasses the distro release system, but then, Youtube also updates itself without the distro's approval, so the software that interfaces with it must as well.
Not to mention "they can download our stuff but have to use a CLI" makes rights holders feel like it's a niche application that can't threaten their bottom line so don't worry about taking steps to prevent it.
What needs to stay under the radar are the dozens of downloader sites and GUI apps that offer the same functionality and ten times the convenience.
I feel the same, but at the same time, it's a similar process as with adblockers; it's an arms race of obfuscation. If you can view a video in your browser, then it principally follows that you can save it to your hard drive. (Similarly, if you're displaying an ad, it follows that you can not display an ad.)
youtube-dl being more known isn't that hazardous, as long as there's a community of people ready to confront the latest ballast sent to make people's lives more miserable - which there is so far, and I'm honestly incredibly grateful for that.
Doubt you can write that in less than 7 lines of non-gibberish. Shell scripts aren't known to be particularly easy to write, read, use and to be portable.
This tool probably works anywhere node runs and the author didn't have to think about that, during the evening they took to write it.
For videos, this should work... added to your .bashrc file, which then needs to be reloaded (only once) by typing source .bashrc in your home directory from the command line:
alias yt='youtube-dl --recode-video mp4'
On my system I have a funky python setup so I need to use this instead:
alias yt='unset PYTHONPATH; youtube-dl --recode-video mp4'
I don't see the need for interactivity because the only format I ever want is mp4.
For audio, I would just do a separate alias with a different name. For example (edit: tested):
alias yta='youtube-dl --extract-audio --audio-format mp3'
Invoke each with the URL. For example:
Video:
yt https://www.youtube.com/watch?v=IFe9wiDfb0E
Audio:
yta https://www.youtube.com/watch?v=qKS8KsTC4IU
If a youtube URL has extra parameters separated by &, delete those before invoking. For example the URL below, which has extra parameters because it came from a playlist:
f=/tmp/youtube-urls;sed 's/^/url=/'|exec curl -K-|exec grep -o "https%3A%2F%2Fr[0-9][a-zA-Z0-9.%_-]*"|exec sed -n 's/%26/\&/g;s/%3D/=/g;s/%3A/:/g;s/%2F/\//g;s/%3F/?/g;s/%25/%/g;w'"$f"'';case $1 in --help|-help|-h|-?)exec echo usage: echo youtube-url\|$0 \[fmt\] \[saveas\];;*) a=$(exec grep "itag=$1"\& $f 2>/dev/null);case ${#a} in 0) echo "the fmt (itag number \""$1"\") is not available. here are the available formats:";exec grep -o "itag=[0-9]*" $f;;*)exec curl -4o ${1-$c} $a;esac;esac
The idea of interactively selecting the format seems like it would become cumbersome after the user learns the formats, i.e., itag numbers, that she can play on her computer.
Once she has that knowledge, she will just be looking for the same itag numbers every time, e.g., 18 or 22 are popular for mp4.
Anyway, shell is more suitable for this kind of programs: you could write a bash script that ties together youtube-dl and dmenu or fzf or whatever within an hour.
How do you define complexity threshold for something being worth sharing?
More to the point, why do you define that threshold? What benefit is there to saying someone's work that they're sharing has to be sufficiently complicated? Simple things can be immensely useful and well worth sharing.
Why would there be a link between trivial and worth sharing?
On the contrary, I find that a lot of tools I'm happy to have are the trivials one.
Who have never used whoami for example ? It's two chained system calls, but I'm still glad it's baked in and I don't have to rewrite it everywhere I use it.
A bash script would be shorter for sure. I just could not find any that does the job. And those 50 line bash scripts tend to be hard to distribute and maintain IMHO.
I think this is probably their first CLI tool. Maybe even their first node project? So that's probably where the excitement comes from. And if that's the case then I can appreciate that.
Personally I'd probably go a similar approach as yourself but that's purely because I don't have node installed.
I struggle to use youtube-dl due to the abundance of parameters, and this gave me the realisation that it is missing interactive mode! So, I'm quite grateful that this person shared their creation.
People gotta realize that Node.js is actually excellent for this sort of task. It hits quite a few sweet spots as an all-round Unix scripting language:
* Familiar syntax and semantics (Bash/Zsh/Fish/Csh/Ksh/Ash/Wtfsh are all way weirder than JS - and they are _only_ scripting languages, while JS is already everywhere)
* Sane packaging, package management, and ecosystem (overreliance on third-party module is a thing exactly because NPM works with much less friction than e.g. the Python situation)
* Asynchronous event model lets you safely do multiple things in parallel without worrying about low-level memory management, or which concurrency boilerplate library to use, etc.
All in all, it's the perfect Unix glue. All it misses is a mature S-expression-based syntax.
For example, take the problem of locking your encrypted partitions before suspending your Linux machine. Since distro developers are too "busy" to provide what one might say is basic functionality, power users took matters in their own hands and wrote this 100-ish line Bash script (https://github.com/vianney/arch-luks-suspend), which evolved into this 1500-ish line Go monstrosity (https://github.com/guns/go-luks-suspend)... as well as these ~75 lines of Node.js (https://github.com/egasimus/node-luks-suspend).
But yeah, sure, hurr durr JaavaaScriipt baad, 0.1+0.2!==0.3, 'true'==true, etc... The horror!
Look, I'm generally with you, there isn't anything particularly bad about modern JavaSscript, and it's a perfectly fine language to write your tools in if that's what you like.
But your points are... Weird.
1. Familiar to people who know JavaScript, sure. But shell is the language you interact with the CLI with, so more people who use the CLI are familiar with a shell.
2. Most languages have a standard library.
3. I know many people who would disagree with you. Anyway, pip works fine in a Unix environment, which is what we're discussing here.
4. We're talking about glue code, right? Glue doesn't tend to be very parallel.
5. So, perfect Unix glue? Sure, you like it, and that's great for you! Keep on doing you. It's obviously capable of making nice tools, but so are most languages...
6. You luks-suspend example is weird because the Node.js code obviously does so much less than the other versions. I have no idea what the Go version is doing, but the shell version does things like remount the filesystem, handle udev, etc... A bash version of the JavaScript code would be shorter!
7. 0.1+0.2!=0.3 is a result of floating point arithmetic being floating point arithmetic. It's not specific to JavaScript.
1. Most people who interact with a shell know how to make it run one command with arguments; some eventually learn about IO redirection. Many such people who use Bash as a "command prompt" would find the way it does even simple things like variables and conditionals exceedingly arcane. Readability is important, and, for better or worse, a compact curly bracket syntax is seen as "the norm".
2. Different languages' standard libraries have different scopes. Node's is just right for the task at hand.
3. I know many people who disagree with me, too. That doesn't make me wrong (or right ;-)).
4. Why not? Doing a task and displaying a rich progress indicator is doing two tasks. Glue is a continuum - for me, the concept includes orchestration of external "heavy lifting" tasks according to the desired logic, as well as processing intermediary outputs in simple ways so the whole pipeline fits together.
5. Well, I've evaluated it and found it to my liking. I guess you grow to like the tools you use to accomplish things, and if everyone who went out of their way to publicly dismiss Node.js instead went out of their way to use it in place of Bash scripting, they might find it to their liking as well.
6. Yet ironically, it has been the most reliable in achieving its purpose of locking my encrypted partitions when my computer goes to sleep, not to mention unlocking them when I wake it up.
7. Exactly - but people keep pulling this and other quirks out of their ass as "proof" JavaScript is some horrible language. Show me a shell scripting language that even has floating point arithmetic! But maybe glue code can't ever need that, either?
> Readability is important, and, for better or worse, a compact curly bracket syntax is seen as "the norm".
Python seems pretty popular.
> I know many people who disagree with me, too. That doesn't make me wrong
The problem isn't that you're saying node is well suited. The problem was your suggestion that node was uniquely well suited when actually the CLI is a solved problem in most languages.
> Well, I've evaluated it and found it to my liking.
That's all that really matters when it comes to your own personal usage. Just don't expect everyone to have those same preferences nor argue with other developers that your preferences are any kind of "fact".
> I guess you grow to like the tools you use to accomplish things, and if everyone who went out of their way to publicly dismiss Node.js instead went out of their way to use it in place of Bash scripting, they might find it to their liking as well.
Personally I don't like node and find Bash easier however it's thankfully not an "either/or" argument where your only options are Javascript or Bash. There's a whole plethora of options out there.
> Yet ironically, it has been the most reliable in achieving its purpose of locking my encrypted partitions when my computer goes to sleep, not to mention unlocking them when I wake it up.
That's a pretty meaningless anecdote because it could be the result of a whole plethora of reasons outside of the language's control (including but not limited to user error) or even completely made up (like your point about floating point arithmetic (see below) or your other comment about one liners being more readable than multi-lined code).
> Show me a shell scripting language that even has floating point arithmetic!
Quite a few do: ksh93, yash, zsh and murex (disclaimer: I wrote that shell) are all interactive shells which support floating point arithmetic.
In terms of shell scripting languages, you can add the following to the above group: awk, bc, dc and perl.
Even those shells that don't support floating point arithmetic can still do pseudo floating point maths by using powers of 10 multipliers and printf to place the decimal point:
printf %.10f\\n "$((1000000000 * 365/7))e-9"
Ok, I'll grant you that is rather ugly. Thankfully you can call other shell scripting languages from within your POSIX/C shell:
bc <<< "scale=10; 365/7"
This isn't a hard thing to research either; which suggests to me you haven't done your homework before making your arguments.
One reason it's so popular: even though it skips the actual curly brackets (thus making itself more accessible), it doesn't stray too far from the general syntactic paradigm of "curly-bracket languages'" - even though I've also seen people being alienated by the idea of "significant whitespace", again for no other reason but that it "seems wrong" to them.
> The problem was your suggestion that node was uniquely well suited
No, just much better suited than people give it credit for. And this ecosystem brings new things to the table, too.
> when actually the CLI is a solved problem in most languages.
Being able to build a CLI at all? I guess, sure. Being able to quickly build an accessible CLI? Hardly so. Everything in the area of HCI is in flux, and gatekeeping the CLI domain for does more harm than good. Btw do you know anyone doing this sort of interface before Yeoman? https://github.com/synox/youtube-dl-interactive/raw/master/d.... before JS.
> There's a whole plethora of options out there.
Which is why you've got to take into account opportunity cost, choice paralysis, etc. All of the options out there are positively fascinating, but if I have the options to solve problems in multiple contexts with the same familiar tool, I tend to usually go for it.
> Quite a few do: ksh93, yash, zsh and murex (disclaimer: I wrote that shell) are all interactive shells which support floating point arithmetic.
None of which are POSIX, or common defaults?
> In terms of shell scripting languages, you can add the following to the above group: awk, bc, dc
None of which are shells - they're DSLs you can call into from the shell.
> perl
Again, not really a shell. Though I wouldn't be surprised if anyone's using it as one.
> bc <<< "scale=10; 365/7" > 365/7
Even though these tools have a good reason to be like this, having to _research_ how to _call out to an external executable_ to do something as trivial as floating point division (in the sense of "cmon man I can do this in my head, why does is it have to be so hard to tell a computer to do it") is hardly on anyone's list of priorities.
At this point, nearly anyone would rather reach for the scripting language they're already familiar with, rather than learn an unfamiliar one just because it has historically dominated the problem domain.
> one liners being more readable than multi-lined code
Admittedly, that was a pretty bullshit argument. Not any less nuanced than "[l]ine counts mean jack shit", either - I take it as a general rule that one would prefer to write less code.
And I'm not weighing any of the examples against "[c]ode golfing, minifified code, shell 1 liners, Perl, etc. I didn't really take into account the extra functionality of the Go version - it was just as unreliable as the Bash one so I quickly discarded it when I found something that worked.
There's a story behind this comparison. It was likely user error, in the sense I failed to prepare my initramfs correctly using mkinitcpio or dracut (incidentally, both of those being shell scripts - insufficiently documented and not very readable to my untrained eye, either) But the JavaScript version worked for me because it didn't even try to chroot into the initramfs - it just moves a tiny handful of required binaries into a ramdisk, and works from that.
Since that solution _works_, I am hard pressed not to commend its minimalism. Is it inconceivable that that the author was not more clever, but the tools that they used enabled them to come up with a simpler solution in the time that would've otherwise been spent googling `bc` or poring over the sources of initramfs builders?
> One reason it's so popular: even though it skips the actual curly brackets (thus making itself more accessible), it doesn't stray too far from the general syntactic paradigm of "curly-bracket languages'" - even though I've also seen people being alienated by the idea of "significant whitespace", again for no other reason but that it "seems wrong" to them.
All you're describing there is structured programming languages. Of course people like structured languages, that's why nearly every single programming language developed in the last 40 years (that isn't designed to interface directly with hardware) is structured. Even languages of entirely different paradigms such as Common Lisp, LOGO, Prolog, MATLAB and Forth all support structured control flows. Even BASIC - for all it's warts - supported structured logic (even if many implementations lacked proper functions).
So you're not making a convincing argument about curly braces now you've redefined the argument as structured control flows.
> No, just much better suited than people give it credit for. And this ecosystem brings new things to the table, too.
Maybe. My personal opinion is the unmanageable nest of unvetted dependencies makes it a worse solution than people give it credit for. As a pure dev tool for personal use, go for it. But I don't want to read 10k+ lines of code in the dependencies of a tool that's ostensibly just an interactive alias.
I get the whole "standing on the shoulders of giants" thing but it's gotten out of hand with node. Half the time even people writing code in node don't know what dependencies their dependencies are pulling in. All that trust placed in an ecosystem that is very easy to abuse (and has been abused too). So no thanks, I don't think that's the right tool for serious CLI work which often needs to be run on trusted machines.
> Being able to build a CLI at all? I guess, sure. Being able to quickly build an accessible CLI? Hardly so.
Actually yes. I've been writing software for 30 year in well over a dozen different languages. Much, not all but a reasonable amount of that, was writing software for servers.
> Everything in the area of HCI is in flux
Not in terms of the CLI. We're using the same standards that were defined in the 60s. If anything, the problem with the CLI is that it's engineered around design principles which are no longer relevant to modern terminals. For example:
* we no longer need to inline formatting with escape sequences - we have better markups for that.
* ASCII is no longer relevant in our international community with different writing languages and character sets.
* Terminals are no longer teletypes - there's no reason why we shouldn't be able to inline multimedia content
* Terminals are no longer teletypes - we shouldn't need to be defining TTYs/PTYs as an OS abstraction
* Defining TTYs is OS specific and a fucking mess on even some of the better designed UNIXes
* Capturing and sending signals can often be OS specific and a fucking mess to work with on even some of the better designed UNIXes
* Not all OSs even support the same suite of signals
* Signal hotkeys can't even be redefined - which means you can't always compensate for one OS missing a signal that another might have. The best you can do is enable or disable the capture of that hotkey on the TTY. Which again requires OS specific syscalls
* POSIX pipes are just byte streams. Which makes working with structured data a fucking nightmare (hence why Powershell, jq and murex all exist)
* stderr is a lousy way of capturing errors
* in fact error handling on the whole is just terrible on POSIX shells (hence why murex exists)
* POSIX shells don't support any kind of unit testing (again, hence why murex exists)
* exit numbers are a half arsed solution
* named pipes are a file system kludge
* TTY modes can get broken if a fork()ed process changes the mode and crashes for forgets to reset it on exit. Meaning your shell then becomes non-functional without manually fork()ing `reset`
* ANSI escape sequences are anything but standardised. With different terminal emulators often doing significantly different things
This is all just off the top of my head so I could likely think of more if I spent any time thinking about it. All of those problems are a result of the command line interface being largely the exact same design as it was in the 60s.
So no, the CLI definitely is not in a state of flux. The exact opposite of that in fact.
More likely is the Javascript libraries you're using for CLI work is in a constant state of flux. This was a problem I had when I used to write software in node and it's one of many personal reasons I decided against using it for anything I really cared about.
> Which is why you've got to take into account opportunity cost, choice paralysis, etc. All of the options out there are positively fascinating, but if I have the options to solve problems in multiple contexts with the same familiar tool, I tend to usually go for it.
That makes no sense. Last week you were saying it couldn't be done and now you're saying the problem is too much choice. Make your mind up.
> [ksh93, yash, zsh and murex] None of which are POSIX, or common defaults?
You didn't say "POSIX shells", you said "shell scripting languages"
Also node isn't a common default either.
> [Perl] Again, not really a shell. Though I wouldn't be surprised if anyone's using it as one.
Again you didn't say "POSIX shells", you said "shell scripting languages". You can very much use Perl as a shell scripting language (and that is in fact still a common use case for it).
> Even though these tools have a good reason to be like this, having to _research_ how to _call out to an external executable_ to do something as trivial as floating point division (in the sense of "cmon man I can do this in my head, why does is it have to be so hard to tell a computer to do it") is hardly on anyone's list of priorities.
Actually computers can't do proper floating point arithmetic (in the sense that humans do) either. That's why a lot of languages (and spreadsheets too) demonstrate seemingly broken logic when comparing floating point numbers.
> Admittedly, that was a pretty bullshit argument. Not any less nuanced than "[l]ine counts mean jack shit", either - I take it as a general rule that one would prefer to write less code.
Line counts literally do mean jack shit. There are plenty of examples of where more lines equal better code. As a developer you should already know this.
All of these features are things other languages have too (I personally disagree that npm's "download the world" model is better than Python's envs). A huge drawback Node has is that it's not installed by default on any Linux distribution I know of, so I'd take a Python script over a Node one any day. Compiled languages have an even better distribution story, and Rust's Cargo is nothing to laugh at.
I'm not sure what you hope your posted comparisons show, but the Go "monstrosity" seems to have many more features than the Node one.
Sure, the the node code uses fewer lines than the Go code. But what you’ve failed to notice is the Go code calls APIs directly rather than fork()ing and does so within its own code base (how many lines of code is in the countless imports of your node example?). Plus the Go code also has unit tests which the others don’t.
Line numbers are always a terrible metric for proving anything but in this case your especially abusing that data.
In absence of more accessible complexity measurements, line counts do matter somewhat. E.g. the briefer a program is, the more readable it is (last time I checked, brainpower and time is much more expensive than computing power and bandwidth.)
The Node example has zero external dependencies outside of the Node.js standard library.
Well, compared to a shell script or a static binary...the Node example requires you to have Node (+ NPM) on your machine and, unless you wrap it all up for homebrew/apt/some other package repo. you're going to be using `npm install -g` to use it as opposed to shoving an executable into your PATH.
All of your stated benefits are nice for the maintainer but mean little to the end user. This gushing praise for Node reads like nobody has ever built elegant CLI apps until Ryan Dahl turned of age and put JS on a server.
> In absence of more accessible complexity measurements, line counts do matter somewhat. E.g. the briefer a program is, the more readable it is (last time I checked, brainpower and time is much more expensive than computing power and bandwidth.)
There are as many exceptions to that rule are there are examples when it’s true. Code golfing, minifified code, shell 1 liners, Perl, etc.
I would argue that even an esoteric, densely packed one-liner can still be more readable than the equivalent program spread out over a greater number of lines - simply because you can keep it all in sight at the same time and work through it without losing the bigger picture.
So you'd like to use APL, then? He didn't say symbols. Verbose code can be much easier, though stupidly verbose is stupid. Compromise is, as usual, the best solution.
I've created this account with my very own purpose, what about you? It is to lurk less, and it's the first account that sticks because this time I remembered to whitelist HN in my cookie autodeleter.
Nevertheless, it is you who choose to engage with my comments on such a dismissive note. I am prepared that my statements will initially be called ridiculous, but speaking one's mind is a great way to learn faster and there's always plenty to catch up with.
HN is a community. Users needn't use their real name, but should have some identity for others to relate to. Otherwise we may as well have no usernames and no community, and that would be a different kind of forum. https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...
Remembering flags is hard, yes. If you use a lot of different apps remembering the flags for something you use infrequently is a pain. I don't think I've ever used ffmpeg without looking at the docs first.
As for shell scripts, they are a thing, and this thing is one (just written in node rather than bash.)
See more node cli tools at: https://github.com/sindresorhus/awesome-nodejs#command-line-...