Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Yash-rs Homepage

Yash-rs is a POSIX-compliant command-line shell for Unix-like operating systems, written in Rust. It is designed to be a modern, fast, and extensible shell that provides a rich set of features for users. Currently, it is in the early stages of development and is not yet feature-complete.

Features

  • Shell scripting with POSIX-compliant syntax and semantics
  • Enhanced scripting features (extension to POSIX)
  • Locale support
  • Job control and minimal interactivity
  • Command-line editing and history
  • Tab completion and suggestions

License

This project is licensed under the GNU General Public License v3.0. The full license text can be found at https://github.com/magicant/yash-rs/blob/master/yash-cli/LICENSE-GPL.

Documentation

Navigate the documentation using the left sidebar. If the sidebar is not visible, you can toggle it by clicking the “hamburger” icon in the top-left corner.

Currently, the documentation is a work in progress and may not cover all features or usage scenarios. Contributions are welcome via issues or pull requests to enhance the documentation.

This documentation, hosted at https://magicant.github.io/yash-rs/, is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0). The full license text can be found at https://github.com/magicant/yash-rs/blob/master/docs/LICENSE.

Installation

Downloading precompiled binaries

Precompiled binaries are available for the following platforms:

  • aarch64-unknown-linux-gnu
  • x86_64-unknown-linux-gnu
  • aarch64-apple-darwin
  • x86_64-apple-darwin

You can download the latest release from the GitHub releases page. Download the appropriate binary for your platform and place it in a directory included in your PATH environment variable.

Building from source

Yash-rs is written in Rust, so you need to have the Rust toolchain installed. The recommended way to install Rust is via rustup.

If you are using Windows Subsystem for Linux (WSL), make sure to install the Linux version of rustup, not the native Windows version. For alternative installation methods, refer to the rustup book.

By default, installing rustup also installs the stable Rust toolchain. If the stable toolchain is not installed, you can add it with the following command:

rustup default stable

To install yash-rs, run:

cargo install yash-cli

Running yash-rs

After installation, you can run yash3 from the command line.

Getting started

Starting the shell

To start the shell, run yash3 from the command line. This starts an interactive shell session.

yash3

You will see a prompt, indicating the shell is ready for commands:

$

Using the shell interactively

Once the shell is running, you can type commands. The shell executes each command you enter, and you will see the output in the terminal.

Most commands run a utility, which is a program that performs a specific task. For example, you can run the echo utility to print a message:

$ echo "Hello, world!"
Hello, world!

In this example, $ is the shell prompt, and echo "Hello, world!" is the command you entered. The shell executed the echo utility, which printed “Hello, world!” to the terminal.

You can also run other utilities, such as ls, which lists the files in the working directory:

$ ls
Documents  Downloads  Music  Pictures  Videos

The output varies depending on the files in your working directory.

Interrupting a command

To interrupt a running command, press Ctrl+C. This sends an interrupt signal to the running utility, causing it to terminate. For example, if you run a command that takes a long time, you can cancel it with Ctrl+C:

$ sleep 10

This command sleeps for 10 seconds, but you can interrupt it by pressing Ctrl+C. This aborts the sleep utility and returns you to the shell prompt immediately.

Note: Some utilities may not respond to Ctrl+C if they are designed to ignore or handle the interrupt signal differently.

Exiting the shell

To exit the shell, use the exit command:

$ exit

This ends the shell session and returns you to your previous shell.

Alternatively, you can press Ctrl+D to exit the shell. This sends an empty command to the shell, causing it to exit.

Running scripts

You can also run scripts in the shell. To do this, create a script file with the commands you want to run. For example, create a file called script.sh with the following content:

echo "This is a script"
echo "Running in the shell"

Run this script in the shell by using the . utility:

$ . ./script.sh
This is a script
Running in the shell

You can also run the script by passing it as an argument to the shell:

$ yash3 ./script.sh
This is a script
Running in the shell

This runs the script in a new shell session. The output will be the same.

If you make the script executable, you can run it directly:

$ chmod a+x script.sh
$ ./script.sh
This is a script
Running in the shell

The chmod utility makes the script file executable. This allows you to run the script directly, without specifying the shell explicitly, as in the previous example.

Note the ./ in the commands above. This indicates that the script is in the current directory. If you omit ./, the shell searches for the script in the directories listed in the PATH environment variable. If the script is not in one of those directories, you will get a “utility not found” error.

Shell language

The shell interprets input as commands written in the shell language. The language has a syntax (how commands are written and structured) and semantics (how commands are executed). This page gives a brief overview of shell commands.

Simple commands

A simple command is a sequence of words that are not reserved words or operators. For example:

ls

or:

echo "Hello, world!"

Most simple commands run a utility—a program that performs a specific task. The first word is the utility name; the rest are arguments.

All words (except redirection operators) in a simple command are expanded before the utility runs. See Words, tokens, and fields for details on parsing and expansion.

You can use parameters to change command behavior dynamically. See the Assignment section for how to define variables.

See the Simple command details section for more on how simple commands work, including word expansion, assignment, and redirection.

Compound commands

Compound commands group commands, control execution, and handle conditions and loops. Examples include if, for, while, and case. Compound commands can contain multiple simple commands and be nested. See the Compound commands section for details.

Pipelines and lists

Pipelines connect the output of one command to the input of another, letting you chain commands. A list is a sequence of commands separated by operators like ;, &&, or ||. See the Pipelines and Lists sections for usage.

Functions

Functions are reusable blocks of code you can define and call in the shell. They help organize scripts and interactive sessions. Functions can take parameters. See the Functions section for details.

Redirections

Redirections control where command input and output go. Use them to save output to files or read input from files. See the File descriptors and redirections section for more.

Aliases

Aliases are shortcuts for longer commands or command sequences. They let you create custom names for commands you use often. See the Aliases section for details.

Words, tokens, and fields

In the shell language, a word is a sequence of characters, usually separated by whitespace. Words represent commands, arguments, and other elements in the shell.

In this example, echo, Hello,, and world! are all words:

$ echo Hello, world!
Hello, world!

The first word (echo) is the name of the utility to run. The other words are arguments passed to that utility.

Before running the utility, the shell expands words. This means the shell processes certain characters and sequences in the words to produce the final command line. For example, $ is used for parameter expansion, letting you access variable values:

$ name="Alice"
$ echo Hello, $name!
Hello, Alice!

Here, $name is expanded to its value (Alice) before echo runs.

To prevent expansion, quote the characters you want to treat literally. For example, to print $name without expanding it, use single quotes:

$ echo '$name'
$name

Tokens and operators

A token is a sequence of characters processed as a single unit in shell syntax. The shell divides input into tokens, which are then parsed to form commands. A token is either a word or an operator.

The shell recognizes these operators:

  • Newline – Command separator
  • ; – Command separator
  • & – Asynchronous command
  • && – Logical and
  • || – Logical or
  • | – Pipeline
  • ( – Start of a subshell
  • ) – End of a subshell
  • ;; – End of a case item
  • ;& – End of a case item
  • ;;& – End of a case item
  • ;| – End of a case item
  • < – Input redirection
  • <& – Input redirection
  • <( – Process redirection
  • << – Here document
  • <<- – Here document
  • <<< – Here string
  • <> – Input and output redirection
  • > – Output redirection
  • >& – Output redirection
  • >| – Output redirection
  • >( – Process redirection
  • >> – Output redirection
  • >>| – Pipeline redirection

When recognizing operators, the shell matches the longest possible sequence first. For example, && is a single operator, not two & operators, and <<<< is recognized as <<< and <, not two << operators.

Blank characters (spaces and tabs) separate tokens unless quoted. Words (non-operator tokens) must be separated by at least one blank character. Operators do not need to be separated by blanks if they are recognized as expected.

These two lines are equivalent:

$ ((echo hello))
hello
$ ( ( echo hello ) )
hello

However, you cannot omit the space between ; and ;; in a case command:

$ case foo in (foo) echo foo; ;; esac
foo
$ case foo in (foo) echo foo;;; esac
error: the pattern is not a valid word token
 --> <stdin>:2:29
  |
2 | case foo in (foo) echo foo;;; esac
  |                             ^ expected a word
  |

Word expansion

The shell performs several kinds of word expansion before running a utility, such as replacing parameters with their values or evaluating arithmetic expressions.

The following expansions happen first:

After these, the shell performs these steps in order:

  1. Field splitting
  2. Pathname expansion
  3. Quote removal

The result is a list of words passed to the utility. Each word resulting from these expansions is called a field.

A subset of these expansions are performed depending on the context. For example, when assigning a variable, the shell performs tilde expansion, parameter expansion, command substitution, arithmetic expansion, and quote removal before the assignment. However, field splitting and pathname expansion do not occur during variable assignment, since the value of a variable cannot be split into multiple fields.

Quoting and escaping

Some characters have special meanings in the shell. For example, the dollar sign ($) is used for parameter expansion, and the asterisk (*) is used for pathname expansion. To include these characters literally in a command, you need to quote or escape them.

What characters need quoting?

The following characters have special meanings in the shell and may need quoting or escaping:

|  &  ;  <  >  (  )  $  `  \  "  '

Whitespace characters (spaces, tabs, and newlines) also need quoting or escaping if they are part of a command word.

Additionally, the following characters are treated specially in certain contexts:

*  ?  [  ]  ^  -  !  #  ~  =  %  {  ,  }

It is best to quote or escape these characters when they are used to stand for themselves in a command. You also need to quote reserved words (e.g., if, while, etc.) to treat them as regular words.

The following subsections explain methods for quoting and escaping characters in the shell.

Single quotes

Single quotes enclose a string and prevent the shell from interpreting special characters. Everything inside single quotes is treated literally, including spaces and special characters.

For example, the following command prints the string "$foo" without interpreting the $ as a parameter expansion or the " as a double quote:

$ echo '"$foo"'
"$foo"

Single quotes can contain newline characters:

$ echo 'foo
> bar'
foo
bar

Note that the > prompt indicates that the command continues on the next line.

You cannot include a single quote character inside a single-quoted string. Use double quotes or a backslash to escape it:

$ echo "'"
'
$ echo \'
'

Double quotes

Double quotes enclose a string. Most characters inside double quotes are treated literally, but some characters are still interpreted by the shell:

  • $: Parameter expansion, command substitution, and arithmetic expansion
  • `: Command substitution
  • \: Character escape, only before ", $, `, \, and newline

For example, single quote characters are treated literally and parameter expansion is performed inside double quotes:

$ foo="*  *"
$ echo "foo='$foo'"
foo='*  *'

Double quotes prevent field splitting and pathname expansion on the result of expansions. If the argument to the echo utility were not double-quoted in the above example, the output might have been different depending on the result of field splitting and pathname expansion.

Backslash

The backslash escapes special characters, allowing you to include them in a string without interpretation.

Outside double quotes, a backslash can escape any character except newline. For example:

cat My\ Diary.txt

This prints the contents of the file My Diary.txt.

When used in double quotes, the backslash only escapes the following characters: ", $, `, and \. For example:

cat "My\ Diary\$.txt"

This will print the contents of the file My\ Diary$.txt. Note that the backslash before the space is treated literally, and the backslash before the dollar sign is treated as an escape character.

When used in a braced parameter expansion that occurs inside double quotes, the backslash additionally escapes }:

$ var="{foo}bar"
$ echo "${var#*\}}"
bar

Within backquotes and arithmetic expansions, backslashes only escape $, `, and \. If backquotes appear inside double quotes, backslashes also escape ". See examples in the Command substitution and Arithmetic expansion sections.

Line continuation

Line continuation allows you to split long commands into multiple lines for better readability. Use a backslash followed by a newline to indicate that the command continues on the next line. A backslash-newline pair is ignored by the shell as if it were not there. Line continuation can be used inside and outside double quotes, but not inside single quotes.

$ echo "This is a long command that \
> continues on the next line"
This is a long command that continues on the next line

To treat a newline literally rather than as a line continuation, use single or double quotes.

Dollar single quotes

Dollar single quotes ($'...') are used to specify strings with escape sequences, similar to those in C. The content inside the quotes is parsed, and recognized escape sequences are replaced with their corresponding characters.

For example, \n is replaced with a newline character:

$ echo $'foo\nbar'
foo
bar

The following escape sequences are recognized inside dollar single quotes:

  • \\ – backslash
  • \' – single quote
  • \" – double quote
  • \n – newline
  • \t – tab
  • \r – carriage return
  • \a – alert (bell)
  • \b – backspace
  • \e or \E – escape
  • \f – form feed
  • \v – vertical tab
  • \? – question mark
  • \cX – control character (e.g., \cA for ^A)
  • \xHH – byte with hexadecimal value HH (1–2 hex digits)
  • \uHHHH – Unicode character with hexadecimal value HHHH (4 hex digits)
  • \UHHHHHHHH – Unicode character with hexadecimal value HHHHHHHH (8 hex digits)
  • \NNN – byte with octal value NNN (1–3 octal digits)

Unrecognized or incomplete escape sequences cause an error.

A backslash followed by a newline is not treated as a line continuation inside dollar single quotes; they are rejected as an error.

Example with Unicode:

$ echo $'\u3042'
あ

Dollar single quotes are useful for specifying strings with special characters.

In the current implementation, escape sequences that produce a byte are treated as a Unicode character with the same value and converted to UTF-8. This means that byte values greater than or equal to 0x80 are converted to two bytes of UTF-8. This behavior does not conform to the POSIX standard and may change in the future.

Quote removal

When a word is expanded, any quotation marks (single quotes, double quotes, or backslashes used for quoting) that were present in the original command are removed. This process is called quote removal.

For example:

$ echo 'Hello, world!' # the single quotes are removed during expansion
Hello, world!

Quote removal only affects quotes that were part of the original input, not those introduced by expansions:

$ x='\*'
$ echo $x # the backslash is not removed because it was introduced by expansion
\*

Reserved words

Some words have special meaning in shell syntax. These reserved words must be quoted to use them literally. The reserved words are:

  • ! – Negation
  • { – Start of a grouping
  • } – End of a grouping
  • [[ – Start of a double bracket command
  • case – Case command
  • do – Start of a loop or conditional block
  • done – End of a loop or conditional block
  • elif – Else if clause
  • else – Else clause
  • esac – End of a case command
  • fi – End of an if command
  • for – For loop
  • function – Function definition
  • if – If command
  • in – Delimiter for a for loop
  • then – Then clause
  • until – Until loop
  • while – While loop

Currently, [[ and function are only recognized as reserved words; their functionality is not yet implemented.

Additionally, the POSIX standard allows for the following optional reserved words:

  • ]] – End of a double bracket command
  • namespace – Namespace declaration
  • select – Select command
  • time – Time command

These four words are not reserved in yash-rs now, but may be in the future.

Where are reserved words recognized?

Reserved words are recognized in these contexts:

  • As the first word of a command
  • As a word following any reserved word other than case, for, or in
  • in as the third word in a for loop or case command
  • do as the third word in a for loop

Examples

This example uses the reserved words for, in, do, and done in a for loop:

$ for i in 1 2 3; do echo $i; done
1
2
3

In the following example, {, do, and } are not reserved words because they are not the first word of the command:

$ echo { do re mi }
{ do re mi }

Reserved words are recognized only when they appear as a whole word. In this example, { and } are not reserved words because they are part of {echo and Hello}:

$ {echo Hello}
error: cannot execute external utility "{echo"
 --> <stdin>:1:1
  |
1 | {echo Hello}
  | ^^^^^ utility not found
  |

To use { and } as reserved words, write them as separate words:

$ { echo Hello; }
Hello

Comments

Use comments to add notes or explanations to shell scripts and commands. The shell ignores comments while parsing commands.

A comment starts with the # character and continues to the end of the line.

$ # This is a comment
$ echo "Hello, world!"  # This prints a message
Hello, world!

Always separate the start of a comment from the preceding word with whitespace. If there is no whitespace, the shell treats the # as part of the word, not as a comment.

$ echo "Hello, world!"# This is not a comment
Hello, world!# This is not a comment

Everything after # on the same line is ignored by the shell. You cannot use line continuation inside comments.

$ echo one # This backslash is not a line continuation 👉 \
one
$ echo two # So this line is a separate command
two

Tilde expansion

In tilde expansion, the shell replaces a tilde (~) at the start of a word with the value of the HOME variable, allowing you to specify paths relative to your home directory. For example, if HOME is /home/alice:

$ HOME=/home/alice
$ echo ~
/home/alice
$ echo ~/Documents
/home/alice/Documents

The HOME variable is usually passed as an environment variable to the shell when the user logs in, so you don’t need to set it manually.

You can also use ~ followed by a username to refer to another user’s home directory:

$ echo ~bob
/home/bob
$ echo ~bob/Documents
/home/bob/Documents

In variable assignments, tilde expansion happens at the start of the value and after each : character:

$ PATH=~/bin:~bob/bin:~clara/bin:/usr/bin
$ echo "$PATH"
/home/alice/bin:/home/bob/bin:/home/clara/bin:/usr/bin

If tilde expansion produces a pathname ending with / followed by another /, one / is removed:

$ HOME=/
$ echo ~/tmp
/tmp

In older shells, //tmp may be produced instead of /tmp, which can refer to a different location. POSIX.1-2024 now requires the behavior shown above.

Tilde expansion only happens at the start of a word, or after each / (or : in variable assignments). If any part of the expansion or delimiter is quoted, the shell treats them literally:

$ echo ~'b'ob
~bob
$ echo ~\/
~/

Currently, the shell ignores any errors during tilde expansion and leaves the tilde as is. This behavior may change in the future.

The shell may support other forms of tilde expansion in the future, e.g., ~+ for the current working directory.

Parameter expansion

Parameter expansion retrieves the value of a parameter when a command is executed. The basic syntax is ${parameter}.

$ user="Alice" # define a variable
$ echo "Hello, ${user}!" # expand the variable
Hello, Alice!

Unset parameters

If a parameter is unset, the shell expands it to an empty string by default.

$ unset user
$ echo "Hello, ${user}!"
Hello, !

If the nounset shell option is enabled, expanding an unset parameter is an error:

$ set -o nounset
$ echo "Hello, ${user}!"
error: cannot expand unset parameter
 --> <stdin>:2:14
  |
2 | echo "Hello, ${user}!"
  |              ^^^^^^^ parameter `user` is not set
  |
  = info: unset parameters are disallowed by the nounset option

Using nounset is recommended to catch typos in variable names.

Omitting braces

Braces are optional if the parameter is:

  • a variable name with only ASCII letters, digits, and underscores (e.g., $HOME, $user)
  • a special parameter (e.g., $?, $#)
  • a single-digit positional parameter (e.g., $1, $2)

For variable names, the shell uses the longest possible name after $, regardless of whether the variable exists:

$ user="Alice"
$ unset username
$ echo "Hello, $username!" # $user is not considered
Hello, !

For positional parameters, only a single digit is used, even if followed by more digits:

$ set foo bar baz # set three positional parameters
$ echo "$12" # $1 expands to the first positional parameter
foo2

Modifiers

Modifiers change the value of a parameter during expansion. Modifiers can only be used in braced expansions, and only one modifier is allowed per expansion.

Length

The length modifier ${#parameter} returns the number of characters in the parameter’s value.

$ user="Alice"
$ echo "Length of user: ${#user}"
Length of user: 5

As an extension, the length modifier can be used with arrays or special parameters * or @, applying the modifier to each element:

$ users=(Alice Bob Charlie)
$ echo "Lengths of users: ${#users}"
Lengths of users: 5 3 7
$ set yellow red green blue # set four positional parameters
$ echo "Lengths of positional parameters: ${#*}"
Lengths of positional parameters: 6 3 5 4

Switch

The switch modifier changes the result based on whether a parameter is set or empty. There are eight forms:

  • ${parameter-word} – Use word if parameter is unset.
  • ${parameter:-word} – Use word if parameter is unset or empty.
  • ${parameter+word} – Use word if parameter is set.
  • ${parameter:+word} – Use word if parameter is set and not empty.
  • ${parameter=word} – Assign word to parameter if unset, using the new value.
  • ${parameter:=word} – Assign word to parameter if unset or empty, using the new value.
  • ${parameter?word} – Error with word if parameter is unset.
  • ${parameter:?word} – Error with word if parameter is unset or empty.

Examples:

$ user="Alice"
$ echo "Hello, ${user-World}!"
Hello, Alice!
$ unset user
$ echo "Hello, ${user-World}!"
Hello, World!

$ unset PATH
$ PATH="/bin${PATH:+:$PATH}"
$ echo "$PATH"
/bin
$ PATH="/usr/bin${PATH:+:$PATH}"
$ echo "$PATH"
/usr/bin:/bin

$ unset user
$ echo "Hello, ${user=Alice}!"
Hello, Alice!
$ echo "Hello, ${user=Bob}!"
Hello, Alice!

$ user="Alice"
$ echo "Hello, ${user?tell me your name}!"
Hello, Alice!
$ unset user
$ echo "Hello, ${user?tell me your name}!"
error: tell me your name
 --> <stdin>:4:14
  |
4 | echo "Hello, ${user?tell me your name}!"
  |              ^^^^^^^^^^^^^^^^^^^^^^^^^ parameter `user` is not set
  |

In all cases, the following expansions are performed on word before use:

For the = and := forms, quote removal is also performed before assignment. Assignment only works for variables, not special or positional parameters.

If word is empty in the ? and :? forms, a default error message is used.

The nounset option does not apply to expansions with a switch modifier.

Trim

The trim modifier removes leading or trailing characters matching a pattern from a parameter’s value. There are four forms:

  • ${parameter#pattern} – Remove the shortest match of pattern from the start.
  • ${parameter##pattern} – Remove the longest match of pattern from the start.
  • ${parameter%pattern} – Remove the shortest match of pattern from the end.
  • ${parameter%%pattern} – Remove the longest match of pattern from the end.

The value is matched against the pattern, and the matching part is removed.

$ var="banana"
$ echo "${var#*a}"
nana
$ echo "${var##*a}"

$ echo "${var%a*}"
banan
$ echo "${var%%a*}"
b

The pattern is expanded before use:

You can quote part or all of the pattern to treat it literally:

$ asterisks="***"
$ echo "${asterisks##*}" # removes the whole value

$ echo "${asterisks##\*}" # removes the first *
**
$ echo "${asterisks##'**'}" # removes the first two *
*

Compatibility

Some modifiers are ambiguous when used with a certain special parameter. Yash and many other shells interpret ${##}, ${#-}, and ${#?} as length modifiers applied to special parameters #, -, and ?, not as switch or trim modifiers applied to #. The POSIX standard is unclear on this point.

The result is unspecified in POSIX for:

  • a length or switch modifier applied to special parameter * or @
  • a trim modifier applied to special parameter #, *, or @

Command substitution

Command substitution expands to the output of a command. It has two forms: the preferred $(command) form and the deprecated backquote form `command`.

For example, this runs dirname -- "$0" and passes its output to cd:

$ cd -P -- "$(dirname -- "$0")"

This changes the working directory to the directory containing the script, regardless of the current directory.

Syntax

The $(…) form evaluates the command inside the parentheses. It supports nesting and is easier to read than backquotes:

$ echo $(echo $(echo hello))
hello
$ echo "$(echo "$(echo hello)")"
hello

In the backquote form, backslashes escape $, `, and \. If backquotes appear inside double quotes, backslashes also escape ". These escapes are processed before the command is run. A backquote-form equivalent to the previous example is:

$ echo `echo \`echo hello\``
hello
$ echo "`echo \"\`echo hello\`\"`"
hello

The $(…) form can be confused with arithmetic expansion. Command substitution is only recognized if the code is not a valid arithmetic expression. For example, $((echo + 1)) is arithmetic expansion, but $((echo + 1); (echo + 2)) is command substitution. To force command substitution starting with a subshell, insert a space: $( (echo + 1); (echo + 2)).

Semantics

The command runs in a subshell, and its standard output is captured. Standard error is not captured unless redirected. Trailing newlines are removed, and the result replaces the command substitution in the command line.

Currently, yash-rs parses the command when the substitution is executed, not when it is parsed. This may change in the future, affecting when syntax errors are detected and when aliases are substituted.

Arithmetic expansion

Arithmetic expansion evaluates an arithmetic expression and replaces it with the result. The syntax is $((expression)).

$ echo $((1 + 2))
3
$ echo $((2 * 3 + 4))
10
$ echo $((2 * (3 + 4)))
14

Arithmetic expansion works in two steps. First, the expression is processed for parameter expansion, nested arithmetic expansion, command substitution, and quote removal. Then, the resulting string is parsed as an arithmetic expression, and the result replaces the expansion.

$ x=2
$ echo $(($x + $((3 * 4)) + $(echo 5)))
19

Variables

The value of variables appearing as parameter expansions does not have to be numeric, but the resulting arithmetic expression must be valid.

$ seven=7
$ var='6 * sev'
$ echo $((${var}en)) # expands to $((6 * seven))
42

$ seven='3 + 4'
$ echo $((2 * $seven)) # expands to $((2 * 3 + 4)), mind the precedence
10
$ echo $((2 * seven))
error: error evaluating the arithmetic expansion
 --> <arithmetic_expansion>:1:5
  |
1 | 2 * seven
  |     ^^^^^ invalid variable value: "3 + 4"
  |
 ::: <stdin>:3:6
  |
3 | echo $((2 * seven))
  |      -------------- info: arithmetic expansion appeared here
  |

Quoting

Backslash escaping is the only supported quoting mechanism in arithmetic expansion. It can escape $, `, and \. However, escaped characters would never produce a valid arithmetic expression after quote removal, so they are not useful in practice.

$ echo $((\$x))
error: error evaluating the arithmetic expansion
 --> <arithmetic_expansion>:1:1
  |
1 | $x
  | ^ invalid character
  |
 ::: <stdin>:1:6
  |
1 | echo $((\$x))
  |      -------- info: arithmetic expansion appeared here
  |

Field splitting

Field splitting breaks a word into fields at delimiters. This happens after parameter expansion, command substitution, and arithmetic expansion, but before pathname expansion and quote removal.

In this example, $flags is split at the space, so ls receives two arguments:

$ flags='-a -l'
$ ls $flags
total 10468
drwxr-xr-x  2 user group    4096 Oct 10 12:34 Documents
drwxr-xr-x  3 user group    4096 Oct 10 12:34 Downloads
drwxr-xr-x  3 user group    4096 Oct 10 12:34 Music
drwxr-xr-x  2 user group    4096 Oct 10 12:34 Pictures
drwxr-xr-x  2 user group    4096 Oct 10 12:34 Videos

Field splitting does not occur if the expansion is quoted:

$ flags='-a -l'
$ ls "$flags"
ls: invalid option -- ' '
Try 'ls --help' for more information.

Field splitting only applies to the results of parameter expansion, command substitution, and arithmetic expansion, not to literals or tilde expansions:

$ HOME='/home/user/My Documents'
$ ls ~
Documents  Downloads  Music  Pictures  Videos
$ ls "$HOME"
Documents  Downloads  Music  Pictures  Videos
$ ls $HOME
ls: cannot access '/home/user/My': No such file or directory
ls: cannot access 'Documents': No such file or directory

Field splitting only happens where words are expected, such as simple command words, for loop words, and array assignments. It does not occur in contexts expecting a single word, like scalar assignments or case patterns.

$ flags='-a -l'
$ oldflags=$flags # no field splitting; oldflags is '-a -l'
$ flags="$flags -r"
$ ls $flags # field splitting; ls receives '-a', '-l', and '-r'
Videos  Pictures  Music  Downloads  Documents
$ flags=$oldflags # again, no field splitting
$ echo "Restored flags: $flags"
Restored flags: -a -l

IFS

Field splitting is controlled by the IFS (Internal Field Separator) variable, which lists delimiter characters. By default, IFS contains a space, tab, and newline. You can change IFS to use different delimiters.

If IFS is unset, the default value is used:

$ unset IFS
$ flags='-a -l'
$ ls $flags
total 10468
drwxr-xr-x  2 user group    4096 Oct 10 12:34 Documents
drwxr-xr-x  3 user group    4096 Oct 10 12:34 Downloads
drwxr-xr-x  3 user group    4096 Oct 10 12:34 Music
drwxr-xr-x  2 user group    4096 Oct 10 12:34 Pictures
drwxr-xr-x  2 user group    4096 Oct 10 12:34 Videos

If IFS is set to an empty string, no splitting occurs:

$ IFS=''
$ flags='-a -l'
$ ls $flags
ls: invalid option -- ' '
Try 'ls --help' for more information.

Each character in IFS is a delimiter. How fields are split depends on whether a delimiter is whitespace or not.

Non-whitespace delimiters split the word at their position and may produce empty fields:

$ IFS=':'
$ values='a:b::c:d'
$ for value in $values; do echo "[$value]"; done
[a]
[b]
[]
[c]
[d]

Empty fields are not produced after a trailing non-whitespace delimiter:

$ IFS=':'
$ values='a:b:'
$ for value in $values; do echo "[$value]"; done
[a]
[b]

Whitespace delimiters also split the word, but do not produce empty fields. Multiple whitespace delimiters in a row are treated as one:

$ IFS=' '
$ values=' a  b   c'
$ for value in $values; do echo "[$value]"; done
[a]
[b]
[c]

Whitespace and non-whitespace delimiters can be combined in IFS:

$ IFS=' :'
$ values='a:b  c : d:  :e  f '
$ for value in $values; do echo "[$value]"; done
[a]
[b]
[c]
[d]
[]
[e]
[f]

Currently, yash-rs only supports UTF-8 encoded text. This does not fully conform to POSIX, which requires handling arbitrary byte sequences.

Empty field removal

During field splitting, empty fields are removed, except those delimited by non-whitespace IFS characters.

$ empty='' space=' '
$ for value in $empty; do echo "[$value]"; done # prints nothing
$ for value in $space; do echo "[$value]"; done # prints nothing

Empty fields are removed even if IFS is empty:

$ IFS=''
$ empty='' space=' '
$ for value in $empty; do echo "[$value]"; done # prints nothing
$ for value in $space; do echo "[$value]"; done # prints one field containing a space
[ ]

To retain empty fields, quote the word to prevent field splitting:

$ empty='' space=' '
$ for value in "$empty"; do echo "[$value]"; done
[]
$ for value in "$space"; do echo "[$value]"; done
[ ]

Pathname expansion (globbing)

Pathname expansion—also known as globbing—lets you use special patterns to match filenames and directories. The shell expands these patterns into a list of matching pathnames.

For example, *.txt matches all files in the current directory ending with .txt:

$ echo *.txt
notes.txt todo.txt

When does pathname expansion happen?

Pathname expansion occurs after field splitting and before quote removal. It only applies to unquoted words containing globbing characters.

If the noglob shell option is set, pathname expansion is skipped.

Pattern syntax

Pathname expansion uses shell patterns. Patterns may include special characters such as *, ?, and bracket expressions. See Pattern matching for a full description of pattern syntax and matching rules.

The following subsections describe aspects of pattern matching specific to pathname expansion.

Unmatched brackets

Unmatched brackets like [a and b]c are treated as literals. However, some shells may treat other glob characters as literals if they are used with unmatched open brackets. To avoid this, make sure to quote unmatched open brackets:

$ echo [a
[a
$ echo \[*
[a [b [c

Subdirectories

Globs do not match / in filenames. To match files in subdirectories, include / in the pattern:

$ echo */*.txt
docs/readme.txt notes/todo.txt

Brackets cannot contain / because patterns are recognized for each component separated by /. For example, the pattern a[/]b only matches the literal pathname a[/]b, not a/b, because the brackets are considered unmatched in the sub-patterns a[ and ]b.

Hidden files

By default, glob patterns do not match files starting with a dot (.). To match hidden files, the pattern must start with a literal dot:

$ echo .*.txt
.hidden.txt

Glob patterns never match the filenames . and .., even if a pattern begins with a literal dot.

$ echo .*
.backup.log .hidden.txt

No matches

If a pattern does not match any files, it is left unchanged. If the shell does not have permission to read the directory, the pattern is also left unchanged.

Summary

  • Globbing expands patterns to matching pathnames.
  • See Pattern matching for pattern syntax and details.
  • Quote or escape glob characters to use them literally.
  • Patterns that match nothing are left unchanged.

Parameters

A parameter is a name-value pair used to store and retrieve data in a shell script. Parameters can be variables, special parameters, or positional parameters.

Parameter expansion retrieves the value of a parameter when the command is executed.

$ name="Alice" # define a variable
$ echo "Hello, $name!" # expand the variable
Hello, Alice!

Variables

Variables are parameters with alphanumeric names that can be assigned values. Use variable assignment to define a variable by specifying a name and value.

Variable names

Variable names can contain letters, digits, and underscores, but cannot start with a digit. Variable names are case-sensitive, so VAR and var are different variables.

It is common to use uppercase letters for exported (environment) variables and lowercase for local variables. This avoids accidentally overwriting environment variables.

According to POSIX.1-2024, only ASCII letters, digits, and underscores are portably accepted in variable names. Many shells allow additional characters. Yash-rs currently accepts Unicode letters and digits in variable names, but this may change in the future.

Defining variables

To define a variable, use the assignment syntax:

$ user=Alice

This creates a variable named user with the value Alice. There must be no spaces around the = sign.

Before a value is assigned, the following expansions are performed, if any:

$ HOME=/home/alice
$ topdir=~/my_project
$ subdir=$topdir/docs
$ echo $subdir
/home/alice/my_project/docs
$ rawdir='~/$user'
$ echo $rawdir
~/$user

Note that field splitting and pathname expansion do not happen during variable assignment.

$ star=* # assigns a literal `*` to the variable `star`
$ echo "$star" # shows the value of `star`
*
$ echo $star # unquoted, the value is subject to field splitting and pathname expansion
Documents  Downloads  Music  Pictures  Videos

See the Simple commands section for more information on assignment behavior.

Environment variables

Environment variables are variables exported to child processes. To export a variable, use the export built-in:

$ export user=Alice
$ sh -c 'echo $user'
Alice

When the shell starts, it inherits environment variables from its parent. These variables are automatically exported to child processes.

Read-only variables

The readonly built-in makes a variable read-only, preventing it from being modified or unset. This is useful for defining constants.

$ readonly pi=3.14
$ pi=3.14159
error: error assigning to variable
 --> <stdin>:2:1
  |
2 | pi=3.14159
  | ^^^^^^^^^^ cannot assign to read-only variable "pi"
  |
 ::: <stdin>:1:10
  |
1 | readonly pi=3.14
  |          ------- info: the variable was made read-only here
  |

Variables are read-only only in the current shell session. Environment variables exported to child processes are no longer read-only.

Local variables

Variables defined by the typeset built-in (without the --global option) are local to the current shell function. Such variables are removed when the function returns.

$ i=0
$ list() {
>   typeset i
>   for i in 1 2 3; do
>     echo "Inside function: $i"
>   done
> }
$ list
Inside function: 1
Inside function: 2
Inside function: 3
$ echo "Outside function: $i"
Outside function: 0

As shown above, the original variable i is hidden by the local variable i inside the function and is restored when the function returns.

Removing variables

The unset built-in removes a variable.

$ user=Alice
$ echo user=$user
user=Alice
$ unset user
$ echo user=$user
user=

Undefined variables by default expand to an empty string. Use the -u shell option to make the shell treat undefined variables as an error.

Reserved variable names

Some variable names are reserved for special purposes. These variables may affect or be affected by the shell’s behavior.

  • CDPATH: A colon-separated list of directories to search in the cd built-in

  • ENV: The name of a file to be sourced when starting an interactive shell

  • HOME: The user’s home directory, used in tilde expansion

  • IFS: A list of delimiters used in field splitting

    • The default value is a space, tab, and newline.

  • LINENO: The current line number in the shell script

    • This variable is automatically updated as the shell executes commands.
    • Currently, yash-rs does not support exporting this variable.

  • OLDPWD: The previous working directory, updated by the cd built-in

  • OPTARG: The value of the last option argument processed by the getopts built-in

  • OPTIND: The index of the next option to be processed by the getopts built-in

  • PATH: A colon-separated list of directories to search for executable files when running external utilities

  • PPID: The process ID of the parent process of the shell

    • This variable is initialized when the shell starts.

  • PS1: The primary prompt string, displayed before each command in interactive mode

    • The default value is $ (a dollar sign followed by a space).

  • PS2: The secondary prompt string, displayed when a command is continued on the next line

    • The default value is > (a greater-than sign followed by a space).

  • PS4: The pseudo-prompt string, used for debugging output

    • The default value is + (a plus sign followed by a space).

  • PWD: The current working directory

    • This variable is initialized to the working directory when the shell starts and updated by the cd built-in when changing directories.

Arrays

Arrays are variables that can hold multiple values.

Defining arrays

To define an array, wrap the values in parentheses in the assignment syntax:

$ fruits=(apple banana cherry)

Accessing array elements

Unfortunately, accessing individual elements of an array is not yet implemented in yash-rs.

To access all elements of an array, just use the array name in parameter expansion:

$ fruits=(apple banana cherry)
$ for fruit in "$fruits"; do echo "$fruit"; done
apple
banana
cherry

Special parameters

Special parameters are predefined parameters that have symbolic names and provide specific information about the shell environment. They are not user-defined variables and cannot be assigned values with the assignment syntax.

Below are the special parameters and their meanings:

  • @: All positional parameters.

    • Expands to all positional parameters as separate fields. Useful for passing all arguments as is to a utility or function.
    • When expanded outside double quotes, the result is subject to field splitting and pathname expansion. To preserve each parameter as a separate field, use "$@". If there are no positional parameters, "$@" expands to zero fields.
    • In contexts where only one field is expected (such as in the content of a here document), @ expands to a single field with all positional parameters joined by the first character of the IFS variable (defaults to space if unset, or no separator if IFS is empty).
    $ set foo 'bar bar' baz # three positional parameters
$ for value in "$@"; do echo "[$value]"; done
[foo]
[bar bar]
[baz]
$ for value in $@; do echo "[$value]"; done
[foo]
[bar]
[bar]
[baz]

  • *: All positional parameters.

    • Similar to @, but in double quotes, * expands to a single field containing all positional parameters joined by the first character of IFS.
    $ set foo 'bar bar' baz # three positional parameters
$ for value in "$*"; do echo "[$value]"; done
[foo bar bar baz]
$ for value in $*; do echo "[$value]"; done
[foo]
[bar]
[bar]
[baz]

  • #: Number of positional parameters.

    $ set foo 'bar bar' baz
$ echo "$#"
3

  • ?: Exit status of the last command.

  • -: Current shell options.

    • Expands to the short names of all currently set shell options, concatenated together. Options without a short name are omitted. For example, if -i and -m are set, the value is im.

  • $: Process ID of the current shell.

    • Set when the shell starts and remains constant, even in subshells.

  • !: Process ID of the last asynchronous command.

    • Updated when an asynchronous command is started or resumed.

  • 0: Name of the shell or script being executed.

    • Set at shell startup and remains constant.
    • If neither the -c nor -s shell option is active, the value of 0 is the first operand in the shell invocation (the script pathname).
    • If the -c option is used and a second operand is present, that operand is used as 0.
    • Otherwise, 0 is set to the first argument passed to the shell, usually the shell’s name.

Positional parameters

Positional parameters are parameters identified by their position in the command line. They are commonly used to pass arguments to scripts or functions.

Initializing positional parameters

Positional parameters are set when the shell starts:

  • If neither the -c nor -s shell option is active, positional parameters are set to the operands after the first operand in the shell invocation. For example:

    yash3 script.sh arg1 arg2 arg3

    Here, the positional parameters are arg1, arg2, and arg3.

  • If the -c option is used, positional parameters are set to operands after the second operand, if any:

    yash3 -c 'echo "$1" "$2"' arg0 arg1 arg2

    The positional parameters are arg1 and arg2. The second operand (arg0) is used as special parameter 0, not as a positional parameter.

  • If the -s option is active, positional parameters are set to all operands in the shell invocation:

    yash3 -s arg1 arg2 arg3

    The positional parameters are arg1, arg2, and arg3.

Modifying positional parameters

To set positional parameters, use the set built-in:

$ set foo bar baz
$ echo "$1" "$2" "$3"
foo bar baz

To append new parameters without removing existing ones, use set -- "$@" followed by the new parameters:

$ set old_param1 old_param2
$ set -- "$@" new_param1 new_param2
$ echo "$1" "$2" "$3" "$4"
old_param1 old_param2 new_param1 new_param2

The -- marks the end of options, so parameters starting with - are not treated as options.

To remove the first N positional parameters, use shift:

$ set foo bar baz qux
$ echo "$1" "$2" "$3" "$4"
foo bar baz qux
$ shift 2
$ echo "$1" "$2"
baz qux

If set is called with no operands, positional parameters are unchanged. To clear them, use set -- or shift "$#".

When a function is called, positional parameters are set to the function’s arguments. You can modify them within the function using set or shift. After the function returns, the original positional parameters are restored.

Expanding positional parameters

In parameter expansion, positional parameters are referenced by their position, starting from 1:

$ set foo bar baz
$ echo "$3" "$2" "$1"
baz bar foo

For positions above 9, use braces:

$ set a b c d e f g h i j k l m n o p q r s t u v w x y z
$ echo "${1}" "${10}" "${26}"
a j z
$ echo "$10" # expands as ${1}0
a0

To expand all positional parameters at once, you can use the special parameter @ or *. Specifically, to pass all positional parameters intact to a utility or function, expand @ in double quotes:

$ set foo 'bar bar' baz
$ printf '[%s]\n' "$@"
[foo]
[bar bar]
[baz]

To get the number of positional parameters, use the special parameter #:

$ set foo bar baz
$ echo "$#"
3

Parsing positional parameters

To parse positional parameters as options and arguments, use the getopts built-in. This is useful for scripts that handle command-line options:

$ set -- -a arg1 -b arg2 operand1 operand2
$ while getopts a:b: opt; do
>   case "$opt" in
>     (a)
>       echo "Option -a with argument: $OPTARG"
>       ;;
>     (b)
>       echo "Option -b with argument: $OPTARG"
>       ;;
>     (*)
>       echo "Unknown option: $opt"
>       ;;
>   esac
> done
Option -a with argument: arg1
Option -b with argument: arg2
$ shift $((OPTIND - 1)) # remove parsed options
$ echo "Remaining operands:" "$@"
Remaining operands: operand1 operand2

Simple commands

Simple commands are the basic building blocks of shell commands. They consist of command line words, assignments, and redirections.

Outline

Despite the name, simple commands can have complex behavior. This section summarizes the main aspects before covering details.

Most simple commands run a utility—a program that performs a specific task. A simple command that runs a utility contains a word specifying the utility name, followed by zero or more words as arguments:

$ echo "Hello!"
Hello!

The words are expanded before the utility runs.

A simple command can assign values to variables. To assign a value, join the variable name and value with an equals sign (=) without spaces:

$ greeting="Hello!"
$ echo "$greeting"
Hello!

If assignments are used with utility-invoking words, they must appear before the utility name and they usually affect only that utility invocation:

$ TZ='America/Los_Angeles' date
Sun Jun  1 06:49:30 AM PDT 2025
$ TZ='Asia/Tokyo' date
Sun Jun  1 10:49:30 PM JST 2025

A simple command can also redirect input and output using redirection operators. For example, to redirect output to a file:

$ echo "Hello!" > output.txt
$ cat output.txt
Hello!

Redirections can appear anywhere in a simple command, but are typically placed at the end.

Syntax

The formal syntax of a simple command, written in Extended Backus-Naur Form (EBNF):

simple_command            := normal_utility - reserved_word, [ normal_argument_part ] |
                             declaration_utility, [ declaration_argument_part ] |
                             assignment_part, [ utility_part ] |
                             "command", [ utility_part ];
utility_part              := normal_utility, [ normal_argument_part ] |
                             declaration_utility, [ declaration_argument_part ] |
                             "command", [ utility_part ];
assignment_part           := ( assignment_word | redirection ), [ assignment_part ];
assignment_word           := ? a word that starts with a literal variable name
                               immediately followed by an unquoted equals sign and
                               optionally followed by a value part ?;
command_name              := word - assignment_word;
declaration_utility       := "export" | "readonly" | "typeset";
normal_utility            := command_name - declaration_utility - "command";
normal_argument_part      := ( word | redirection ), [ normal_argument_part ];
declaration_argument_part := ( assignment_word | word - assignment_word | redirection ),
                             [ declaration_argument_part ];

Key points:

  • A simple command cannot start with a reserved word unless it is quoted.
  • An assignment word must start with a non-empty variable name, but the value can be empty.
  • To treat a word containing = as a command name, quote the variable name or the equals sign.
  • Redirections can appear anywhere in a simple command.

There must be no space around the equals sign in an assignment word. If you need spaces in the value, quote them:

$ greeting="Hello, world!"
$ echo "$greeting"
Hello, world!

The utility names export, readonly, and typeset are declaration utilities; when used as a command name, following argument words are parsed as assignment words if possible, or as normal words otherwise. This affects how arguments are expanded. The utility name command is also special; it delegates to the next word the determination of whether it is a declaration utility or a normal utility. (More utility names may be treated as declaration utilities in the future.)

Semantics

A simple command is executed by the shell in these steps:

  1. Command words (name and arguments) are expanded in order.
    • If the command name is a declaration utility, argument words that look like assignment words are expanded as assignments; the rest are expanded as normal words. Otherwise, all arguments are expanded as normal words.
    • Command words are expanded before assignments, so assignments do not affect command words in the same command.
    • Expansions in assignments and redirections are not yet performed in this step.
    • The result is a sequence of words called fields.
    • If expansion fails, the error is reported and the command is aborted.
  2. Redirections are performed, in order.
    • If there are any fields, redirections are processed in the current shell environment. If a redirection fails, the error is reported and the command is aborted.
    • If there are no fields, redirections are processed in a subshell. In this case, redirections do not affect the current shell, and errors are reported but do not abort the command.
  3. Assignments are performed, in order.
    • Each assignment value is expanded and assigned to the variable in the current shell environment. See Defining variables for details.
    • Assigned variables are exported if there are any fields or if the allexport option is enabled.
    • If an assigned variable is read-only, the error is reported and the command is aborted.
  4. If there are any fields, the shell determines the target to execute based on the first field (the command name), as described in Command search below.
  5. The shell executes the target:
    • If the target is an external utility, it is executed in a subshell with the fields as arguments. If the execve call used to execute the target fails with ENOEXEC, the shell tries to execute it as a script in a new shell process.
    • If the target is a built-in, it is executed in the current shell environment with the fields (except the first) as arguments.
    • If the target is a function, it is executed in the current shell environment. When entering a function, positional parameters are set to the fields (except the first), and restored when the function returns.
    • If no target is found, the shell reports an error.
    • If there was no command name (the first field), nothing is executed.

Assigned variables are removed unless the target was a special built-in or there were no fields after expansion, in which case the assignments persist. Redirections are canceled unless the target was the exec special built-in (or the command built-in executing exec), in which case the redirections persist.

Command search determines the target to execute based on the command name (the first field):

  1. If the command name contains a slash (/), it is treated as a pathname to an executable file target, regardless of whether the file exists or is executable.
  2. If the command name is a special built-in (like exec or exit), it is used as the target.
  3. If the command name is a function, it is used as the target.
  4. If the command name is a built-in other than a substitutive built-in, it is used as the target.
  5. The shell searches for the command name in the directories listed in the PATH variable. The first matching executable regular file is a candidate target.
    • The value of PATH is treated as a sequence of pathnames separated by colons (:). An empty pathname in PATH refers to the current directory. For example, in the simple command PATH=/bin:/usr/bin: ls, the shell searches for ls in /bin, then /usr/bin, and finally the current directory.
    • If PATH is an array, each element is a pathname to search.
  6. If a candidate target is found:
    • If the command name is a substitutive built-in (like echo or pwd), the built-in is used as the target.
    • Otherwise, the executable file is used as the target.
  7. If no candidate target is found, the command search fails.

An executable file target is called an external utility.

Exit status

  • If a target was executed, the exit status of the simple command is the exit status of the target.
  • If there were no fields after expansion, the exit status is that of the last command substitution in the command, or zero if there were none.
  • If the command was aborted due to an error in expansion, redirection, assignment, or command search, the exit status is non-zero: 127 if command search failed, or 126 if the target could not be executed (e.g., unsupported file type or permission denied).

Pipelines

A pipeline is a sequence of commands connected by pipes (|). The output of each command is passed as input to the next, allowing you to chain commands for more complex tasks.

Basic usage

The syntax for a pipeline is:

command1 | command2 | command3 ...

For example, to list files and filter the output:

$ mkdir $$ && cd $$ || exit
$ > foo.txt > bar.txt > baz.png
$ ls | grep .txt
bar.txt
foo.txt

The | operator may be followed by linebreaks for readability:

$ mkdir $$ && cd $$ || exit
$ > foo.txt > bar.txt > baz.png
$ ls |
> grep .txt |
> wc -l
2

Line continuation can also be used to split pipelines across multiple lines:

$ mkdir $$ && cd $$ || exit
$ > foo.txt > bar.txt > baz.png
$ ls \
> | grep .txt \
> | wc -l
2

If a pipeline contains only one command, the shell runs that command directly. For multiple commands, the shell creates a subshell for each and connects them with pipes. Each command’s standard output is connected to the next command’s standard input. The first command’s input and the last command’s output are not changed. All commands in the pipeline run concurrently.

The shell waits for all commands in the pipeline to finish before proceeding. The exit status of the pipeline is the exit status of the last command in the pipeline. (In the future, yash-rs may only wait for the last command to finish.)

Negation

You can negate a pipeline using the ! reserved word:

$ mkdir $$ && cd $$ || exit
$ ! ls | grep .zip
$ echo $?
0

This runs the pipeline and negates its exit status: if the status is 0 (success), it becomes 1 (failure); if non-zero (failure), it becomes 0 (success). This is useful for inverting the result of a command in a conditional.

Negation applies to the pipeline as a whole, not to individual commands. To negate a specific command, use braces:

$ mkdir $$ && cd $$ || exit
$ ls | { ! grep .zip; } && echo "No zip files found"
No zip files found

Since ! is a reserved word, it must appear as a separate word:

$ !ls | grep .zip
error: cannot execute external utility "!ls"
 --> <stdin>:1:1
  |
1 | !ls | grep .zip
  | ^^^ utility not found
  |

Compatibility

POSIX requires that a pipeline waits for the last command to finish before returning an exit status, and it is unspecified whether the shell waits for all commands in the pipeline to finish. yash-rs currently waits for all commands, but this may change in the future.

POSIX allows commands in a multi-command pipeline to be run in the current shell environment rather than in subshells. Korn shell and zsh run the last command in the current shell environment, while yash-rs runs all commands in subshells.

Some shells like Korn shell and mksh assign special meanings to the ! reserved word immediately followed by the ( operator. For maximum compatibility, ! and ( should be separated by a space.

Grouping

A grouping command combines multiple commands so they are treated as a single command. This is useful for running several commands together in a pipeline or an and-or list.

Braces

Commands grouped in braces { ... } run in the current shell environment.

$ { echo "Hello"; echo "World"; }
Hello
World

A group can span multiple lines:

$ {
> echo "Hello"
> echo "World"
> }
Hello
World

Since { and } are reserved words, they must appear as separate words. See examples in the Keywords section.

Braces are especially useful for treating several commands as a single unit in pipelines or and-or lists:

$ { echo "Hello"; echo "World"; } | grep "Hello"
Hello

$ HOME=$PWD
$ test -f ~/cache/file || { mkdir -p ~/cache; > ~/cache/file; }

Subshells

Commands grouped in parentheses ( ... ) run in a subshell—a copy of the current shell environment. Changes made in a subshell do not affect the parent shell.

$ greeting="Morning"
$ (greeting="Hello"; echo "$greeting")
Hello
$ echo "$greeting"
Morning

Since ( and ) are operators, they can be used without spaces.

Compatibility

Some shells treat two adjacent ( characters specially. For best compatibility, separate open parentheses with a space to nest subshells:

$ ( (echo "Hello"))
Hello

Exit status and conditionals

This section describes the exit status of commands and how to use it to control the flow of execution in the shell.

Exit status

The exit status of a command is a number that indicates an abstract result of the command’s execution. It is used to determine whether a command succeeded or failed.

The value of the exit status is a non-negative integer, typically in the range of 0 to 255. The exit status is stored in the special parameter ? immediately after a command runs.

$ test -f /nonexistent/file
$ echo $? # the exit status of `test`
1
$ echo $? # the exit status of the previous `echo`
0

Before running a command, the initial value of ? is 0.

While the exact meaning of exit status values is specific to each command, there are some common conventions:

  • An exit status of 0 indicates success.
  • A non-zero exit status indicates failure or an error condition. The specific value can provide additional information about the type of failure.
  • Exit statuses in the range of 1 to 125 are generally used by commands to indicate various types of errors or conditions.
  • Exit statuses 126 and greater are reserved by the shell for special purposes.

The following exit statuses are used by the shell to indicate specific conditions:

  • Exit status 126 indicates that a command was found but could not be executed.
  • Exit status 127 indicates that a command was not found.
  • Exit status 128 indicates that the shell encountered an unrecoverable error reading a command.
  • When a command is terminated by a signal, the exit status is 384 plus the signal number. For example, if a command is terminated by SIGINT (signal number 2), the exit status will be 386.

Compatibility of exit statuses

POSIX specifies exit statuses for certain conditions, but there are still many conditions for which POSIX does not define exact exit statuses. Different shells and commands may use different exit statuses for the same conditions, so it’s important to check the documentation of the specific command you are using. Specifically, the exit status of a command terminated by a signal may vary between shells as POSIX only specifies that the exit status must be greater than 128.

Yash-rs internally handles exit statuses as 32-bit signed integers, but receives only the lower 8 bits from child processes running a subshell or external utility. This means that exit statuses that are not in the range of 0 to 255 are truncated to fit into this range. For example, an exit status of 256 becomes 0, and an exit status of 1000 becomes 232.

Exit status of the shell

When exiting a shell, the exit status of the shell itself is determined by the exit status of the last command executed in the shell. If no commands have been executed, the exit status is 0.

If the exit status of the last command indicates that the command was terminated by a signal, the shell sends the same signal to itself to terminate. The parent process (which may or may not be a shell) will observe that the shell process was terminated by a signal, allowing it to handle the termination appropriately. Specifically, if the parent process is also yash, the value of the special parameter ? in the child shell process is reproduced in the parent shell process without modification.

This signal-passing behavior is not supported by all shells; in shells that do not support it, the lower 8 bits of the exit status are passed to the parent process instead. The parent process is likely to interpret this as an ordinary exit status, which may not accurately reflect the original command’s termination by a signal.

The true and false utilities

The true and false utilities simply return an exit status of 0 and 1, respectively. They are often used as placeholders in conditional statements or loops. See the examples in the And-or lists section below.

And-or lists

An and-or list is a sequence of commands that are executed based on the success or failure of previous commands. It allows you to control the flow of execution based on the exit status of commands. An and-or list consists of commands separated by && (and) or || (or) operators. The && operator executes the next command only if the previous command succeeded (exit status 0), while the || operator executes the next command only if the previous command failed (non-zero exit status).

$ test -f /nonexistent/file && echo "File exists" || echo "File does not exist"
File does not exist

Unlike many other programming languages, the && and || operators have equal precedence with left associativity in the shell language:

$ false && echo foo || echo bar
bar
$ { false && echo foo; } || echo bar
bar
$ false && { echo foo || echo bar; }

$ true || echo foo && echo bar
bar
$ { true || echo foo; } && echo bar
bar
$ true || { echo foo && echo bar; }

The && and || operators can be followed by linebreaks for readability:

$ test -f /nonexistent/file &&
> echo "File exists" ||
> echo "File does not exist"
File does not exist

Line continuation can also be used to split and-or lists across multiple lines:

$ test -f /nonexistent/file \
> && echo "File exists" \
> || echo "File does not exist"
File does not exist

The exit status of an and-or list is the exit status of the last command executed in the list.

If commands

An if command is a conditional command that executes a block of commands based on the exit status of a test command. It allows you to perform different actions depending on whether a condition is true or false.

The minimal form of an if command uses the if, then, and fi reserved words that surround commands:

$ mkdir $$ && cd $$ && > foo.txt > bar.txt || exit
$ if diff -q foo.txt bar.txt; then echo "Files are identical"; fi
Files are identical

For readability, each reserved word can be on a separate line:

$ mkdir $$ && cd $$ && > foo.txt > bar.txt || exit
$ if diff -q foo.txt bar.txt
> then
>     echo "Files are identical"
> fi
Files are identical

You can also use the elif reserved word to add additional conditions:

$ if [ -f /dev/tty ]; then
>     echo "/dev/tty is a regular file"
> elif [ -d /dev/tty ]; then
>     echo "/dev/tty is a directory"
> elif [ -c /dev/tty ]; then
>     echo "/dev/tty is a character device"
> fi
/dev/tty is a character device

The else reserved word can be used to provide a default action if none of the conditions are met:

$ file=/nonexistent/file
$ if [ -e "$file" ]; then
>     echo "$file exists"
> elif [ -L "$file" ]; then
>     echo "$file is a symbolic link to a nonexistent file"
> else
>     echo "$file does not exist"
> fi
/nonexistent/file does not exist

The exit status of an if command is the exit status of the last command executed in the then or else clause. If no condition is met and there is no else clause, the exit status is 0 (success).

For repeating commands depending on a condition, see While and until loops.

The errexit option

By default, the shell continues running commands even if one fails (returns a non-zero exit status). This can cause later commands to run when they shouldn’t. If you enable the errexit shell option, the shell will exit immediately when any command fails, stopping further execution.

$ set -o errexit # or: set -e
$ test -e /dev/null
$ echo "Ok, continuing..."
Ok, continuing...
$ test -e /nonexistent/file
$ echo "This will not be printed"

In this example, after test -e /nonexistent/file fails, the shell exits right away, so you won’t see any more prompts or output.

The errexit option only applies to the result of pipelines. It is ignored in these cases:

Although errexit does not catch every error, it is recommended for scripts to avoid unexpected results from failed commands. To skip errexit for a specific command, append && true:

$ set -o errexit
$ test -e /nonexistent/file && true
$ echo "The exit status was $?"
The exit status was 1

Pattern-based branching

The case command performs pattern matching on a value and executes commands for the first matching pattern. This is useful for branching logic based on specific values or patterns.

Case command basics

A case command begins with the case reserved word, followed by the value to match. After the in reserved word, each branch specifies a pattern in parentheses, followed by a block of commands. Each block ends with ;;, and the command ends with esac.

For example, this command matches the value of foo and runs the corresponding commands:

$ case foo in
> (foo)
>     echo "Matched foo"
>     ;;
> (bar)
>     echo "Matched bar"
>     ;;
> esac
Matched foo

Patterns

Patterns can use wildcards and bracket expressions for flexible matching. For example, to match any string starting with f:

$ case foo in
> (f*)
>     echo "Starts with f"
>     ;;
> (b*)
>     echo "Starts with b"
>     ;;
> esac
Starts with f

To match multiple patterns, separate them with a pipe |:

$ case foo in
> (foo|bar)
>     echo "Matched foo or bar"
>     ;;
> esac
Matched foo or bar

Word expansion

Both the value and patterns undergo word expansion:

$ value="Hello" pattern="[Hh]*"
$ case $value in
> ($pattern)
>     echo "Matched pattern"
>     ;;
> esac
Matched pattern

The value is always expanded first. Patterns are expanded only when the shell needs to match them. Once a pattern matches, remaining patterns are not expanded.

Quote special characters in values or patterns to avoid unwanted expansion or matching:

$ case ? in
> ('?')
>     echo "Matched a single question mark"
>     ;;
> (?)
>     echo "Matched any single character"
>     ;;
> esac
Matched a single question mark

Continuing to the next branch

Instead of ;;, use ;& to continue execution with the next branch, regardless of whether its pattern matches. This allows multiple branches to run in sequence:

$ case foo in
> (foo)
>     echo "Matched foo"
>     ;&
> (bar)
>     echo "Matched bar, or continued from foo"
>     ;;
> (baz)
>     echo "Matched baz"
>     ;;
> esac
Matched foo
Matched bar, or continued from foo

Use ;;& or ;| to continue pattern matching in subsequent branches, so commands in multiple matching branches can run:

$ case foo in
> (foo)
>     echo "Matched foo"
>     ;;&
> (bar)
>     echo "Matched bar"
>     ;;
> (f*)
>     echo "Matched any string starting with f"
>     ;;
> esac
Matched foo
Matched any string starting with f

The ;;& and ;| terminators are extensions to POSIX. yash-rs supports both, but other shells may support only one or neither.

Miscellaneous

If no branch matches, or there are no branches, the shell skips the case command without error.

Use * as a catch-all pattern:

$ case foo in
> (bar)
>     echo "Matched bar"
>     ;;
> (*)
>     echo "Matched anything else"
>     ;;
> esac
Matched anything else

Use '' or "" as an empty value or pattern:

$ case "" in
> ('')
>     echo "Matched empty string"
>     ;;
> esac
Matched empty string

The opening parenthesis ( can be omitted if the first pattern is not literally esac, but parentheses are recommended for clarity:

$ case foo in
> foo)
>     echo "Matched foo"
>     ;;
> esac
Matched foo

Branches can have empty command blocks:

$ case bar in
> (foo)
>     ;;
> (bar)
>     echo "Matched bar"
>     ;;
> esac
Matched bar

The ;; terminator can be omitted for the last branch:

$ case foo in
> (foo)
>     echo "Matched foo"
>     ;;
> (bar)
>     echo "Matched bar"
> esac
Matched foo

Exit status

The exit status of case is that of the last command executed in the last executed branch. If the last executed branch has no commands, or no pattern matches, the exit status is 0.

Formal syntax

The formal syntax of the case command, in Extended Backus-Naur Form (EBNF):

case_command := "case", word, { newline }, "in", { newline },
                { branch }, [ last_branch ], "esac";
newline      := "\n";
branch       := pattern_list, branch_body, terminator, { newline };
last_branch  := pattern_list, branch_body;
pattern_list := "(", word, { "|" , word }, ")"
              | (word - "esac"), { "|" , word }, ")";
branch_body  := { newline }, [ list, [ newline, branch_body ] ];
terminator   := ";;" | ";&" | ";;&" | ";|";

Loops

Loops repeatedly execute a sequence of commands, either by iterating over a list or while a condition holds. They are useful for automating repetitive tasks or processing multiple items.

For loops

A for loop iterates over a list of strings, executing a block of commands for each string. In each iteration, the string is assigned to a variable for use in the commands.

$ for user in alice bob charlie; do
>     echo "Hello, $user!"
> done
Hello, alice!
Hello, bob!
Hello, charlie!

The word after for is the loop variable, assigned to each string in the list after in. The do reserved word starts the command block, and done ends the loop.

The semicolon after the list is optional if do is on a new line:

$ for user in alice bob charlie
> do
>     echo "Hello, $user!"
> done
Hello, alice!
Hello, bob!
Hello, charlie!

The in reserved word can also be on a separate line:

$ for user
> in alice bob charlie; do
>     echo "Hello, $user!"
> done
Hello, alice!
Hello, bob!
Hello, charlie!

Word expansion is performed on the list:

$ for file in *.txt; do
>     echo "$file contains $(wc -l -- "$file") lines"
>     echo "First line: $(head -n 1 -- "$file")"
> done
file1.txt contains 10 lines
First line: This is the first line of file1.
file2.txt contains 5 lines
First line: This is the first line of file2.

If the list is empty, the loop does not run:

$ for user in; do
>     echo "Hello, $user!"
> done

If in and the list are omitted, the loop iterates over the positional parameters as if in "$@" were specified:

$ set alice bob charlie
$ for user do
>     echo "Hello, $user!"
> done
Hello, alice!
Hello, bob!
Hello, charlie!

The exit status of a for loop is the exit status of the last command run in the loop, or 0 if the loop does not run.

While and until loops

A while loop executes commands as long as a condition is true. An until loop is similar, but continues until the condition becomes true. The do reserved word separates the condition from the loop body, and done ends the loop.

$ count=1
$ while [ $count -le 5 ]; do
>     echo "Count: $count"
>     count=$((count + 1))
> done
Count: 1
Count: 2
Count: 3
Count: 4
Count: 5

$ count=1
$ until [ $count -gt 3 ]; do
>     echo "Count: $count"
>     count=$((count + 1))
> done
Count: 1
Count: 2
Count: 3

See Exit status and conditionals for details on how exit status affects loop conditions.

The exit status of a while or until loop is that of the last command run in the loop body, or 0 if the loop body does not run. Note that the exit status of the condition does not affect the exit status of the loop.

Break and continue

The break utility exits the current loop. The continue utility skips to the next iteration.

$ for i in 1 2 3 4 5; do
>     if [ $i -eq 3 ]; then
>         echo "Breaking at $i"
>         break
>     fi
>     echo "Iteration $i"
> done
Iteration 1
Iteration 2
Breaking at 3

$ for i in 1 2 3 4 5; do
>     if [ $i -eq 3 ]; then
>         echo "Skipping $i"
>         continue
>     fi
>     echo "Iteration $i"
> done
Iteration 1
Iteration 2
Skipping 3
Iteration 4
Iteration 5

By default, break and continue affect the innermost loop. You can specify a numeric operand to affect the n’th outer loop:

$ for i in 1 2; do
>     for j in a b c; do
>         if [ "$j" = "b" ]; then
>             echo "Breaking outer loop at $i, $j"
>             break 2
>         fi
>         echo "Inner loop: $i, $j"
>     done
> done
Inner loop: 1, a
Breaking outer loop at 1, b

Pattern matching

This section describes the pattern matching notation used in the shell. Patterns are used in pathname expansion, case commands, and parameter expansion modifiers.

Literals

A literal is a character that matches itself. For example, the pattern a matches the character a. All characters are literals except for the special characters described below.

Quoting

Quoting makes a special character behave as a literal. See the Quoting section for details. Additionally, for unquoted parts of a pattern produced by parameter expansion, command substitution, or arithmetic expansion, backslashes escape the following character, but such backslashes are not subject to quote removal.

In this example, no pathname expansion occurs because the special characters are quoted:

$ echo a\*b
a*b
$ asterisk='*'
$ echo "$asterisk"
*
$ quoted='a\*b'
$ echo $quoted
a\*b

Special characters

The following characters have special meanings in patterns:

  • ? – Matches any single character.
  • * – Matches any number of characters, including none.
  • [...] – Matches any single character from the set of characters inside the brackets. For example, [abc] matches a, b, or c. Ranges can be specified with a hyphen, like [a-z] for lowercase letters.
  • [!...] and [^...] – Matches any single character not in the set of characters inside the brackets. For example, [!abc] matches any character except a, b, or c.

The [^...] form is not supported in all shells; prefer using [!...] for compatibility.

$ echo ?????? # prints all six-character long filenames
Videos
$ echo Do* # prints all files starting with Do
Documents Downloads
$ echo [MP]* # prints all files starting with M or P
Music Pictures
$ echo *[0-9] # prints all files ending with a digit
foo.bak.1 foo.bak.2 bar.bak.3

Special elements in brackets

Bracket expressions […] can include special elements:

  • Character classes: [:class:] matches any character in the specified class. Available classes:

    • [:alnum:] – Alphanumeric characters (letters and digits)
    • [:alpha:] – Alphabetic characters (letters)
    • [:blank:] – Space and tab characters
    • [:cntrl:] – Control characters
    • [:digit:] – Digits (0-9)
    • [:graph:] – Printable characters except space
    • [:lower:] – Lowercase letters
    • [:print:] – Printable characters including space
    • [:punct:] – Punctuation characters
    • [:space:] – Space characters (space, tab, newline, etc.)
    • [:upper:] – Uppercase letters
    • [:xdigit:] – Hexadecimal digits (0-9, a-f, A-F)
    $ echo [[:upper:]]* # prints all files starting with an uppercase letter
Documents Downloads Music Pictures Videos
$ echo *[[:digit:]~] # prints all files ending with a digit or tilde
foo.bak.1 foo.bak.2 bar.bak.3 baz~

  • Collating elements: [.char.] matches the collating element char. A collating element is a character or sequence of characters treated as a single unit in pattern matching. Collating elements depend on the current locale and are not yet implemented in yash-rs.

  • Equivalence classes: [=char=] matches the equivalence class of char. An equivalence class is a set of characters considered equivalent for matching purposes (e.g., a and A in some locales). This feature is not yet implemented in yash-rs.

Locale support is not yet implemented in yash-rs. Currently, all patterns match the same characters regardless of locale. Collating elements and equivalence classes simply match the characters as they are, without any special treatment.

Special considerations for pathname expansion

See the Pathname expansion section for additional rules that apply in pathname expansion.

Arithmetic expressions

Arithmetic expressions in arithmetic expansion are similar to C expressions. They can include numbers, variables, operators, and parentheses.

Numeric constants

Numeric constants can be:

  • Decimal, written as-is (e.g., 42)
  • Octal, starting with 0 (e.g., 042)
  • Hexadecimal, starting with 0x or 0X (e.g., 0x2A)
$ echo $((42))   # decimal
42
$ echo $((042))  # octal
34
$ echo $((0x2A)) # hexadecimal
42

All integers are signed 64-bit values, ranging from -9223372036854775808 to 9223372036854775807.

C-style integer suffixes (U, L, LL, etc.) are not supported.

Floating-point constants are not supported, but may be added in the future.

Variables

Variable names consist of Unicode alphanumerics and ASCII underscores, but cannot start with an ASCII digit. Variables must have numeric values.

$ a=5 b=10
$ echo $((a + b))
15

If a variable is unset and the nounset shell option is off, it is treated as zero:

$ unset x; set +o nounset
$ echo $((x + 3))
3

If the nounset option is on, an error occurs when trying to use an unset variable:

$ unset x; set -o nounset
$ echo $((x + 3))
error: cannot expand unset parameter
 --> <arithmetic_expansion>:1:1
  |
1 | x + 3
  | ^ parameter `x` is not set
  |
 ::: <stdin>:2:6
  |
2 | echo $((x + 3))
  |      ---------- info: arithmetic expansion appeared here
  |
  = info: unset parameters are disallowed by the nounset option

If a variable has a non-numeric value, an error occurs.

$ x=foo
$ echo $((x + 3))
error: error evaluating the arithmetic expansion
 --> <arithmetic_expansion>:1:1
  |
1 | x + 3
  | ^ invalid variable value: "foo"
  |
 ::: <stdin>:2:6
  |
2 | echo $((x + 3))
  |      ---------- info: arithmetic expansion appeared here
  |

Currently, variables in arithmetic expressions must have a single numeric value. In the future, more complex values may be supported.

Operators

The following operators are supported, in order of precedence:

  1. ( ) – grouping
  2. Postfix:
    • ++ – increment
    • -- – decrement
  3. Prefix:
    • + – no-op
    • - – numeric negation
    • ~ – bitwise negation
    • ! – logical negation
    • ++ – increment
    • -- – decrement
  4. Binary (left associative):
    • * – multiplication
    • / – division
    • % – modulus
  5. Binary (left associative):
    • + – addition
    • - – subtraction
  6. Binary (left associative):
    • << – left shift
    • >> – right shift
  7. Binary (left associative):
    • < – less than
    • <= – less than or equal to
    • > – greater than
    • >= – greater than or equal to
  8. Binary:
    • == – equal to
    • != – not equal to
  9. Binary:
    • & – bitwise and
  10. Binary:
    • | – bitwise or
  11. Binary:
    • ^ – bitwise xor
  12. Binary:
    • && – logical and
  13. Binary:
    • || – logical or
  14. Ternary (right associative):
    • ? : – conditional expression
  15. Binary (right associative):
    • = – assignment
    • += – addition assignment
    • -= – subtraction assignment
    • *= – multiplication assignment
    • /= – division assignment
    • %= – modulus assignment
    • <<= – left shift assignment
    • >>= – right shift assignment
    • &= – bitwise and assignment
    • |= – bitwise or assignment
    • ^= – bitwise xor assignment

Other operators, such as sizeof, are not supported.