Practicals to Introduction to UNIX, summer semester 2015/16

Petr Kučera, KTIML MFF UK

Basic information

Useful links

Credit requirements

At the end of each practical homeworks shall be assigned, I will expect to receive their solution by e-mail before the beginning of the next practical. If by the end of the semester you have at least 2/3 of possible number of points, you get a credit. If you have at least 1/3 of possible number of points (but less than 2/3), you will have to finish additional homeworks to achieve the credit (number and complexity of these additional homeworks will be in proportional to missing number of points). Each set of homeworks assigned after a practical will be worth 3 points, at the beginning of the semester that could mean three exercises while at the end it can be just one more difficult. Whoever is not able to achieve at least 1/3 will not get a credit from me.

If not stated otherwise, you are allowed to use only the commands, tools and structures we have used on practicals before the homework was assigned (including the one to which homework was assigned). In some cases the tools might be even more restricted. If I come across a solution which is just a copy of another one I will assign no points for this solution. If this case repeats there will be no possibility to obtain a credit from me.

Please, send me your solutions via an e-mail and do so before the beginning of the next practical. The homework assignments are appended below a list of exercises we have done at the corresponding practical (i.e. the current homework exercises are always at the bottom of this web page).

There have been 13 homeworks assigned and thus the number of points required to get the credit is 26. The homeworks assigned to 14th practical are considered as additional and do not count into the limit.

Some peculiarities of the Malá Strana UNIX lab

I think it is worth to know and bear in mind some specifics of the Malá Strana UNIX lab. To Czech students I recommend to check the web pages of the UNIX lab, but as far as I know there is no English version (yet) of these web pages. So I will warn you of the lab properties as we come across them. In this section I would to mention at least the most apparent of them:

  1. Users home directories are placed in a remote AFS filesystem, where standard UNIX permissions do not apply and the standard UNIX commands for file permissions manipulations (mainly chmod) cannot be used or they can be used only in a very restricted way (there is a web page about the ACL which is the system in use, but it is only in Czech to the best of my knowledge).
  2. This means that when you try the chmod command or anything which has some connection to the standard UNIX permission system, you should do so in local directories (such as /tmp in which you are allowed more or less anything). There the standard UNIX permissions apply.
  3. As user accounts are stored remotely, local file /etc/passwd does not contain information about your account or accounts of other lab users. If you want to get your and other users account info in the format of /etc/passwd you can achieve that using a getent command as follows:
    getent passwd
    for other parameters and general help on this command use getent --help.
  4. The same applies to /etc/group and getent group.
  5. The find command (still most probably) is not in the best of terms with AFS filesystem — when used on network AFS drives in the lab such as your home dir you can get unexpected results. You can use find to search on local directories such as /tmp, /etc, /usr etc. The exercises usually suggest it.

Content of the practicals

1st practical (26th February, 2016)

Basic commands, shell command line edit.

More detailed:

Exercises:

  1. In your home directory create a directory named DIR
  2. Copy all files whose filenames satisfy the following conditions to ~/DIR. The files are in /usr/include directory, their names start with m, end with .h and contain a number.
  3. Create a subdirectory called SUBDIR in your DIR directory.
  4. The first five lines of each file you have copied from /usr/include copy to file ~/DIR/SUBDIR/firstfive.
  5. The last lines of files in ~/DIR copy to file ~/DIR/SUBDIR/last.
  6. Concatenate the two files in ~/DIR/SUBDIR into one file ~/DIR/SUBDIR/firstandlast
  7. Delete the files in ~/DIR/SUBDIR except firstandlast.
  8. Store the number of files and directories in ~/DIR into a file ~/DIR/SUBDIR/count
  9. Output the long information on ~/DIR/SUBDIR directory. (Not its content, but information on it).
  10. Delete the contents of ~/DIR/SUBDIR/firstandlast file without removing the file itself.
  11. Add a line containing just a star sign (i.e. *) to file ~/DIR/SUBDIR/firstandlast.
  12. Delete ~/DIR together with all the files it contains.
  13. (Show solutions to preceding exercises)

    Deprecated: Function create_function() is deprecated in /home/kucerap/public_html/include/geshi.php on line 4698
    #!/bin/sh

    # In your home directory create a directory named DIR
    mkdir ~/DIR

    # Copy all files whose filenames satisfy the following conditions to
    # ~/DIR. The files are in /usr/include directory,
    # their names start with 'm', end with '.h' and contain a
    # number.
    cp /usr/include/m*[0-9]*.h ~/DIR

    # Create a subdirectory called SUBDIR in your DIR directory.
    mkdir ~/DIR/SUBDIR

    # The first five lines of each file you have copied from
    # /usr/include copy to file ~/DIR/SUBDIR/firstfive.
    head -n 5 ~/DIR/m*[0-9]*.h >~/DIR/SUBDIR/firstfive

    # The last lines of files in ~/DIR copy to file
    # ~/DIR/SUBDIR/last.
    tail -n 1 ~/DIR/m*[0-9]*.h >~/DIR/SUBDIR/last

    # Concatenate the two files in ~/DIR/SUBDIR into one file
    # ~/DIR/SUBDIR/firstandlast
    cat ~/DIR/SUBDIR/* >~/DIR/SUBDIR/firstandlast

    # Delete the files in ~/DIR/SUBDIR except firstandlast.
    rm ~/DIR/SUBDIR/firstfive
    rm ~/DIR/SUBDIR/last

    # Store the number of files and directories in ~/DIR into a
    # file ~/DIR/SUBDIR/count
    ls ~/DIR | wc -l >~/DIR/SUBDIR/count

    # Output the long information about ~/DIR/SUBDIR directory.
    # (Not its content, but information about it).
    ls -ld ~/DIR/SUBDIR

    # Delete the contents of ~/DIR/SUBDIR/firstandlast file
    # without removing the file itself.
    cp /dev/null ~/DIR/SUBDIR/firstandlast

    # Add a line containing just a star sign (i.e. *) to file
    # ~/DIR/SUBDIR/firstandlast.
    echo '*' >>~/DIR/SUBDIR/firstandlast

    # Delete ~/DIR together with all files the it contains.
    rm -r ~/DIR
  14. Output lines number 11-20 from file /etc/passwd.
    (Show solution)
    #!/bin/sh

    head -n20 /etc/passwd | tail -n10

Homeworks:

  1. Write a command which copies all files from /usr/include whose names start with a lower case letter and contain a digit to your home directory. [1 point]
  2. Write a command which outputs the number of groups in the system (number of lines of /etc/group). [1 point]
  3. Write a command which removes all files and directories (together with subtrees) with names starting with a dot from the current working directory (I suggest not to try this in your home directory). [1 point]

2nd practical (3rd March, 2016)

Basic commands and shell usage.

Commands: date, tee, touch, ssh, scp, who, who am i, id, mesg, talk, mail (mailx), last (not a standard command).

Exercises:

  1. Create a file with name “-f” which contains a line with current date and time. Then delete this file.
    (Show solution)
    #!/bin/sh

    # Create a file -f with date and time:
    date >-f
    # Delete file -f:
    rm -- -f
  2. Create a file named “fixed date” (i.e. “fixed<space>date”) with last date and time of modification is set to 13:30, 1st February, 2009.
    (Show solution)
    #!/bin/sh

    touch -t 200902011330 "fixed date"
    # or
    touch -d "2009-02-01 13:30:00" "fixed date"
  3. Copy the file /etc/passwd into your home directory. The copy should have name accounts and the last modification date and time should be preserved, i.e. should be the same as the last modification date and time of /etc/passwd.
    (Show solution)
    #!/bin/sh

    cp -p /etc/passwd ucty
  4. Set the time and date of last modification of file accounts to the same time as the last modification time of the file /etc/group.
    (Show solution)
    #!/bin/sh

    touch -r /etc/group accounts
  5. Write a command which outputs the list of files in directory /usr/bin sorted lexicographically in decreasing order. The output is written to files bina, binb, and to the screen. The screen output is viewed in pages (use less, or more).
    (Show solution)
    #!/bin/sh

    ls -r /usr/bin | tee bina binb | more
  6. What happens during the execution of the following three commands? What is the difference betweeen them?
    1. mv file /dev/null
    2. cp file /dev/null
    3. cat file >/dev/null
  7. Write commands that output the ten biggest and the ten smallest files in directory /etc.
    (Show solution)
    #!/bin/sh

    ls -S /etc | head
    ls -rS /etc | head
  8. Find a file in /usr/bin with the most recent modification date and time.
    (Show solution)
    #!/bin/sh

    ls -lrt /usr/bin | head -n 1
  9. Write a command which outputs your current UID.
    (Show solution)
    #!/bin/sh

    id -u
  10. Write a command that outputs the list of names of groups to which you belong as a user.
    (Show solution)
    #!/bin/sh

    id -Gn
  11. Send the contents of file /etc/group by an e-mail using command mail (or mailx).
    (Show solution)
    #!/bin/sh

    mail -s "Obsah /etc/group" adresa </etc/group

Homeworks:

  1. Write a command which sets the last modification date and time of a file to 11th March, 1993, 15:04. [1 point]
  2. Write a command which determines the number of groups to which you belong. [1 point]
  3. Write a command which outputs the name of second largest file in the directory /usr/bin. [1 point]

3rd practical (10th March, 2016)

Permissions and simple utilities.

Commands: chmod, chgrp, chown, file, ln, df, du, umask.

Attention, in Malá Strana UNIX lab try permissions in /tmp directory. Other directories are on afs file system which has a different system of permission handling.

Exercises:

  1. Try the command df which determines amount of space used on a disk. Try different options (such as -k and -P, you can also try the other ones but note that only other standard parameter is -t).
  2. Try the command du on directory /tmp or on your home directory with different options (-k, -s, -x), find out more about these options in the specification.
  3. In your home directory create a link to /etc/passwd. First try to create a hard link, then a soft link. Why a hard link cannot be created? Call ls -l to find out how a soft link is displayed in its output.
    (Show solution)
    #!/bin/sh

    # Hard link
    ln /etc/passwd ~
    # Soft link
    ln -s /etc/passwd ~
  4. In directory /tmp create a hard link to another file in /tmp (you can copy some file to /tmp for this purpose, the hard link will naturally have a different name). Run ls -l on both files and observe how a number in the second column changed. Using ls -i check that both copies (the original file and its hard link) have the same inode number.
  5. Create a simple script and assign it an execute permission.
    (Show solution)
    #!/bin/sh

    cat >script.sh <<EOF
    #!/bin/sh

    echo Hello world!
    EOF


    chmod a+x script.sh
  6. Create a file which the owner cannot neither execute, read, or write to it, the members of file group can read and write to the file, and the others can just execute the and not read nor write to it.
    (Show solution)
    #!/bin/sh
    cat >script.sh <<EOF
    #!/bin/sh

    echo Hello world!
    EOF


    chmod 0761 /tmp/script.sh

    # Equivalently:
    chmod u+rwx,g=u-x,o=u-rw /tmp/script.sh

    # or
    chmod u=rwx,g=rw,o=x /tmp/script.sh
  7. Create a script which cannot be read but has execute permissions. Try tu run this script. Why it is not possible to execute an unreadable shell script?
    (Show solution)
    #!/bin/sh
    cat >/tmp/script.sh <<EOF
    #!/bin/sh

    echo Hello world
    EOF


    chmod a+x-r /tmp/script.sh
    /tmp/script.sh
    # If a shell should run the script, it must be allowed to read it, too.
  8. Create a directory to which anyone can go (by cd, can be used in path) and add files but only the directory owner can list its contents.
    (Show solution)
    #!/bin/sh

    mkdir /tmp/directory
    chmod 0755 /tmp/directory

    # Or equivalently
    chmod u+rwx,go=u-w /tmp/directory
  9. Create a directory to which you can go, you cannot read its contents but you can add files to it. Create a file in this directory which can be only read (and not written to or executed). Delete this file using rm. Why you can delete a file which you cannot modify?
    (Show solution)
    #!/bin/sh
    mkdir /tmp/directory
    chmod 0300 /tmp/directory
    # Or equivalently
    chmod u+wx-r,go-rwx /tmp/directory

    touch /tmp/directory/file
    chmod 0400 /tmp/directory/file
    # Or equivalently
    chmod u+r-wx,go-rwx /tmp/directory/file

    rm -f /tmp/directory/file
  10. Make a copy of /etc/passwd in /tmp using cp without and then with -p option. Observe the difference (with the help of ls -l).
  11. Try setuid bit on a script. Here is how you can do it (with a friend):
    1. One user (A) creates a text file /tmp/file with some text inside and sets the permissions so that its owner can read and write to this file, others cannot do anything like that.
    2. User A creates a script /tmp/script.sh, and sets its setuid and its permission so that user B can run it.
    3. The script /tmp/script.sh contains a command cat /tmp/file
    4. User B logs into the same computer (using ssh) on which user A created afformentioned file and script.
    5. User B tries to output the contents of /tmp/file by first using cat directly and then indirectly by calling /tmp/script.sh.
    6. User B can also try to run sh /tmp/script.sh. In which cases the file contents will be successfully displayed?
    (Show solution)
    #!/bin/sh
    # Commands of user A

    echo Hello >/tmp/file
    echo '#!/bin/sh' >/tmp/script.sh
    echo cat /tmp/file >/tmp/script.sh

    chmod 0600 /tmp/file
    chmod 4755 /tmp/script.sh
    # equivalently
    # chmod u=rw,go= /tmp/file
    # chmod u=rwxs,go=rx /tmp/script.sh
     
  12. Run vimtutor

Homeworks:

  1. Write a command, which adds a group read permission to all files in a subtree of the directory /tmp/dir. [1 point]
  2. Create a file and set its permissions so that the owner can execute this file, read and write to it, group members can execute and read it, and the others can only read the file. In chmod command use a symbolic way of specifying the permissions [1 point] and also the octal numeric way [1 point].

4th practical (17th March, 2016)

Simple utilities.

Commands: diff, cut, paste, tr, sort.

Exercises:

  1. Download file calories.csv, it is a file in a CSV format with semicolon (“;”) used as a field separator.
  2. Replace quotation marks (") with apostrophes (') in file calories.csv
    (Show solution)
    #!/bin/sh

    tr \" \' <calories.csv >/tmp/calories.a.csv
    mv /tmp/calories.a.csv calories.csv
  3. Remove apostrophes (') from file calories.csv. In this and the several following exercises we assume that the file is has been modified by preceding exercises, i.e. here after the previous exercise, quotation marks have been replaced with apostrophes.
    (Show solution)
    #!/bin/sh

    tr -d \' <calories.csv >/tmp/calories.a.csv
    mv /tmp/calories.a.csv calories.csv
  4. Select the first column from file calories.csv, i.e. the column with food names.
    (Show solution)
    #!/bin/sh

    cut -f1 -d\; calories.csv
  5. In file calories.csv change capital letters to small letters, but only in the first column. (Cut the first columns using cut, replace capital letters with small ones using tr, then put them back using paste.)
    (Show solution)
    #!/bin/sh

    cut -f1 -d\; calories.csv >/tmp/first
    cut -f2- -d\; calories.csv >/tmp/others
    tr '[:upper:]' '[:lower:]' </tmp/first | paste -d\; - /tmp/others >calories.csv
    rm /tmp/first /tmp/others
  6. In file calories.csv replace commas (,) with hyphens (-), using diff then determine which lines have been changed.
    (Show solution)
    #!/bin/sh

    tr "," "-" <calories.csv | diff calories.csv -
  7. Sort file calories.csv according to amount of calories in increasing order. The first header line has to remain at the beginning of the file.
    (Show solution)
    #!/bin/sh

    # Save the header
    head -n 1 calories.csv >/tmp/header
    # Sort the rest and append it to the header
    tail -n +2 calories.csv | sort -t\; -k4,4nr >>/tmp/header
    mv /tmp/header calories.csv
  8. Sort file calories.csv according to triple [protein, carbohydrates, fat] in an increasing order. The first header line has to remain at the beginning of the file.
    (Show solution)
    #!/bin/sh

    # Save the header
    head -n 1 calories.csv >/tmp/header
    # Sort the rest and append it to the header
    tail -n +2 calories.csv | sort -t\; -k7,7n -k6,6n -k5,5n >>/tmp/header
    mv /tmp/header calories.csv
  9. Determine the number of different units used in the second column of file calories.csv. First consider two units with different number as different, then as the same, such as in case of “1 Cup” and “2 cup”.
    (Show solution)
    #!/bin/sh

    # In the first solution we consider two units with a different amount
    # as different, e.g. “1 Cup” and “2 Cup” are different units.
    tail -n +2 calories.csv | sort -t\; -k2,2 -u | wc -l

    # If we wish to consider two units with different amounts as the same
    # unit (i.e. we do not want to differentiate between “1 Cup” and “2
    # Cup”, we can proceed in the following way:
    tail -n +2 calories.csv | cut -f2 -d\; | cut -f2- -d" " | sort -u | wc -l
  10. Reorder the last three columns of file calories.csv in the opposite order, i.e. after reordering the first column contains amount of proteins, the second contains amount of carbohydrates and the third column contains amount of fat. Use cut and paste.
    (Show solution)
    #!/bin/sh

    cut -f5 -d\; calories.csv >/tmp/fifth
    cut -f6 -d\; calories.csv >/tmp/sixth
    cut -f7 -d\; calories.csv >/tmp/seventh
    cut -f1-4 -d\; calories.csv | paste -d\; - /tmp/seventh /tmp/sixth /tmp/fifth >/tmp/novy
    mv /tmp/novy calories.csv
    rm /tmp/fifth /tmp/sixth /tmp/seventh
  11. Select the 9 characters with permissions from the output of ls -l command.
    (Show solution)
    #!/bin/sh

    ls -l | cut -c2-10
  12. Suppose you have three files, summand1, summand2 and sum, each of these files contains the same number of lines, each line contains just a number. Compose these files to just one output where each line has the following format:
    summand1+summand2=sum.
    (Show solution)
    #!/bin/sh

    paste -d+= summand1 summand2 sum
  13. Output the list of files in the current directory in a form of “size filename”. The list should be created by filtering the outputs of ls -l and ls commands.
    (Show solution)
    #!/bin/sh

    ls -l | tail -n +2 | tr -s " " | cut -d" " -f5 >/tmp/lsl
    ls | paste -d" " /tmp/lsl -
    rm /tmp/lsl
  14. Output the logins from file /etc/passwd, each line contains 5 logins separated with a comma (,).
    (Show solution)
    #!/bin/sh

    cut -d: -f1 </etc/passwd | paste -d"," - - - - -

    # Alternatively and equivalently:
    cut -d: -f1 </etc/passwd | paste -s -d",,,,\n" -

Homeworks:

  1. In file calories.csv replace each food name with a “-” (minus) character. Other columns should remain unchanged. [1 point]
  2. For each user in /etc/passwd output a pair uid:login. [1 point].
  3. Determine the number of different countries which appear in file ip-by-country.csv. [1 point]

5th practical (24th March, 2016)

Simple utilities.

Commands: comm, join, split, find.

Exercises:

  1. Download files countrycodes_en.csv and kodyzemi_cz.csv. Both of these files are in a CSV format with semicolon (;) used as a separator.
  2. Use files countrycodes_en.csv and kodyzemi_cz.csv to create a list of countries with two columns separated with an equality character (=) where each line has the form of
    Czech name=English name.
    (Show solution)
    #!/bin/sh

    tr -d '"' <kodyzemi_cz.csv | tail -n +2 | sort -t \; -k1,1 >/tmp/kody_s
    tr -d '"' <countrycodes_en.csv | tail -n +2 | sort -t \; -k4,4 >/tmp/codes_s
    join -t\; -1 1 -2 4 -o 1.4,2.1 /tmp/kody_s /tmp/codes_s | tr \; = >translation
    rm /tmp/kody_s /tmp/codes_s
  3. Write the names of files which ocur in /usr/bin but which are not in /bin.
    (Show solution)
    #!/bin/sh

    ls /usr/bin | sort >/tmp/usr_bin
    ls /bin | sort | comm -12 - /tmp/usr_bin
    rm /tmp/usr_bin
  4. Write the numbers of groups which are used in /etc/group but which are not used in /etc/passwd. That is find the numbers of groups which are not used as a primary group for any user.
    (Show solution)
    #!/bin/sh

    cut -d: -f4 </etc/passwd | sort > /tmp/uid
    cut -d: -f3 </etc/group | sort | comm -23 - /tmp/uid
    rm /tmp/uid
  5. Split a file (e.g. /etc/passwd) to parts consisting of five lines, then concatenate these parts back to one file.
    (Show solution)
    #!/bin/sh

    # We shall assume that the number of lines in /etc/passwd is not too
    # big, in particular that when using split the two letter suffix is
    # enough.
    split -l5 /etc/passwd passwdpart
    # The concatenated file will be stored in the current directory, we
    # naturally do not have the write permission to /etc directory.
    cat passwdpart* >passwd
    rm passwdpart*
  6. Write the logins from /etc/passwd to ten lines, the logins on each line are separated with spaces.
    (Show solution)
    #!/bin/sh

    # We assume that /etc/passwd does not contain too much records and
    # that the default two letter suffix is enough in split.
    cut -d":" -f1 /etc/passwd | split -l10 - logins
    paste -d" " logins*
    rm logins*
  7. Write a command which lists all files and directories visible to you in the subtree of current directory, whose names contain substring 'bin'.
    (Show solution)
    #!/bin/sh

    find . -name "*bin*" "(" -type d -o -type f ")
  8. Write a command which lists the number of directories visible to you in the subtree of /etc directory.
    (Show solution)
    #!/bin/sh

    find /etc -type d | wc -l
  9. List all symbolic (or soft) links in a directory /usr. List only links directly present in the directory, not deeper in its subtree.
    (Show solution)
    #!/bin/sh

    # The following can be quite slow

    find /usr -type l '!' -path "/usr/*/*"

    # or faster

    find /usr/* -prune -type l

    # or even better

    find /usr \( -path "/usr/*/*" -prune \) -o \( -type l -print \)
  10. List the files with at least three hard links pointing to them in the subtree in directory /usr/bin.
    (Show solution)
    #!/bin/sh

    find /usr/bin -links +3
  11. Find all files in subtree of directory /tmp, which have size bigger than 100kB and which are readable by anyone.
    (Show solution)
    #!/bin/sh

    find /tmp -size +200 -perm -a+r

Homeworks:

  1. Based on files countrycodes_en.csv and kodyzemi_cz.csv list the names of states which are the same in English and in Czech. [1 point]
  2. Find the files in the subtree of current working directory which have size 0B. [1 point]
  3. Find the files in the subtree of /usr/include the files whose name start with “std” but do not end with “.h”. [1 point]

6th practical (31st March, 2016)

Commands: find, xargs.

Exercises:

  1. In the subtree of /usr/include find the files with which are in depth at least 2 and at most 3 in the subtree.
    (Show solution)
    #!/bin/sh

    find /usr/include -path "/usr/include/*/*" ! -path "/usr/include/*/*/*/*"

    # A more effective solution with prune:
    find /usr/include -path "/usr/*/*/*" -prune -o -path "/usr/*/*"
  2. In the subtree of directory /etc find all files which are newer than /etc/passwd.
    (Show solution)
    #!/bin/sh

    find /etc -newer /etc/passwd
  3. In the subtree of directory /bin find all files which are owned by root and which can be executed by the owner and group members, but not by others.
    (Show solution)
    #!/bin/sh

    find /bin -user root -perm -ug+x ! -perm -o+x
  4. Find all files in directory /etc which belong to group stunnel and write long information using ls -l on each of these files.
    (Show solution)
    #!/bin/sh

    find /etc -group stunnel -exec ls -l {} +
  5. Try the difference between echo $PATH; echo "$PATH"; echo '$PATH';
  6. Try the difference between echo *; echo "*"; echo '*';
  7. List the paths contained in variable $PATH so that each of them will be written on a separate line and there will be no separators “:”.
    (Show solution)
    #!/bin/sh

    echo $PATH | tr ":" "\n"
  8. Output the long information on all directories contained in variable $PATH (not their contents, just the long info).
    (Show solution)
    #!/bin/sh

    echo $PATH | tr ":" "\n" | xargs -I{} ls -ld {}
  9. Create a file with the following three lines:
    a b c
    d e f
    g h i
    
    and try to run the following commands on this file (here named “file”):
    cat file | xargs echo
    cat file | xargs echo -
    cat file | xargs -n 2 echo -
    cat file | xargs -I{} echo "-{}-" "-{}-"
    cat file | xargs -I{} -n 2 echo "-{}-" "-{}-"
    
    Observe what happens when different options are passed to xargs.
  10. Split the file /etc/passwd to parts consisting of five lines each and then reorder these parts in the opposite order (that is assuming /etc/passwd has n lines where n is divisible by five, the lines are now ordered as: n-4, n-3, n-2, n-1, n, n-9, n-8, n-7, n-6, n-5, ..., 1, 2, 3, 4, 5).
    (Show solution)
    #!/bin/sh

    split -l5 /etc/passwd passwdparts
    ls -r passwdparts* | xargs cat >passwd
    rm passwdparts*
  11. Find out which directories stored in variable $PATH contain a program named awk.
    (Show solution)
    #!/bin/sh

    echo $PATH | tr ':' '\n' | xargs -I{} find {}/awk -prune 2>/dev/null

Homeworks:

  1. Find all files in the subtree of /etc which are not newer than /etc/group. [1 point]
  2. Output the list of lines which contains the first line of each file in /usr/include (only the ones in the directory itself, not the ones in subdirectories). The list should not contain any lines more than the first ones (simply call head on each file separately). [1 point]
  3. For each user from /etc/passwd write a line
    <login>:<groups in which the user appears>
    (Hint: You can run id for each user and paste it with the list of the users. ) [1 point]

7th practical (7th April, 2016)

Commands: grep.

  1. From calories.csv select lines in which the food name contains at least three letters.
    (Show solution)
    #!/bin/sh

    grep '^"\([^;]*[[:alpha:]]\)\{3\}";' calories.csv
  2. From file calories.csv select lines in which the food name contains a character which is not alphanumeric (i.e. does not belong to [:alnum:]).
    (Show solution)
    #!/bin/sh

    grep '^"[^;]*[^[:alnum:];][^;]*";' calories.csv

    # Případně

    grep -v '^"[[:alnum:]]*";' calories.csv
  3. From file calories.csv select lines in which amount is measured in units “Piece” or “Cake”.
    (Show solution)
    #!/bin/sh

    grep -e '^[^;]*;[[:digit:].]* Piece;' \
         -e '^[^;]*;[[:digit:].]* Cake;' calories.csv

    # Případně

    grep -E '^[^;]*;[[:digit:].]* (Piece|Cake);' calories.csv
  4. Find files in the subtree of /usr/include whose names have length 5-7 characters not including suffix which is required to by “.h”. (Use grep on the output of find command.)
    (Show solution)
    #!/bin/sh

    find /usr/include -name "*.h" | grep '/[^/]\{5,7\}.h'
  5. Select lines from the output of ls -l in which owner has the same permissions as group members and as others. (Create a file satisfying these condition in say /tmp if necessary.)
    (Show solution)
    #!/bin/sh

    ls -l | grep '^.\(...\)\1\1'
  6. From file countrycodes_en.csv select lines in which the two letter country code is the same as the first two letters of the three letter country code.
    (Show solution)
    #!/bin/sh

    grep '^[[:digit:]]*;\(..\).;\1;' countrycodes_en.csv
  7. Now select from file countrycodes_en.csv those lines in which the two letter country code is not the same as the first two letters of the three letter country code.
    (Show solution)
    #!/bin/sh

    grep -v '^[[:digit:]]*;\(..\).;\1;' countrycodes_en.csv
  8. Find files in the subtree of /usr/include which contain a string “printf” in them.
    (Show solution)
    #!/bin/sh

    find /usr/include -exec grep -q printf {} \; -print

    # or

    find /usr/include | xargs grep -l printf

    # or

    find /usr/include -exec grep -l printf {} +

Homeworks:

  1. In /etc/passwd find all lines where the login shell is /bin/sh. [1 point]
  2. In an HTML file (take this web page for example) titles are enclosed by a pair of <hi>, </hi> tags, where i is a digit 1 to 6. Find all lines with this pair of tags (the digit has to be same in both the opening and closing tag). [1 point]
  3. Output a list of names of files in directory /usr/bin about which command file tells you that they are shell scripts (i.e. output of file on such file contains “POSIX shell script” as a file description). The output of file has form file name: description. [1 point]

8th practical (14th April, 2016)

Commands: sed, read, compound commands (for, while, if, case, etc.), printf, expr, test, dirname, basename.

Other: Variables and expansion, redirection, positional and special parameters/variables ($#, $0, $n, ${n}, shift, "$@", set -, $?, $$, $!), command combination using &&, command grouping and subshell invokation, back apostrophes and $() expansion.

  1. Using sed select only the column with the file size from the output of ls -l.
    (Show solution in sed)
    #!/bin/sh

    ls -l | sed '1d; s/^\([^ ]* *\)\{5\} .*$/\1/'

    (Show solution in ed)
    #!/bin/sh

    # Backslash in front of END prevents shell from performing any
    # expansions in the text before END.

    ls -l > /tmp/lsl.$$
    ed /tmp/lsl.$$<<\END
    2,$g/^/s/^\([^ ]* *\)\{5\} .*$/\1/
    %p
    Q
    END
    rm /tmp/lsl.$$
  2. Using sed transform the lines of /etc/passwd to the form of uid (login).
    (Show solution in sed)
    #!/bin/sh

    sed 's/^\([^:]*\):[^:]*:\([^:]*\).*$/\2 (\1)/' /etc/passwd

    (Show solution in ed)
    #!/bin/sh

    # Backslash in front of END prevents shell from performing any
    # expansions in the text before END.

    # If wish to store the results into the input file,
    # we can add commmands w and q.

    cp /etc/passwd /tmp/passwd.$$
    ed /tmp/passwd.$$<<\END
    g/^/s/^\([^:]*\):[^:]*:\([^:]*\).*$/\2 (\1)/
    %p
    Q
    END
    rm /tmp/passwd.$$
  3. Using sed put character # to the beginning of lines 10 to 20 in file /etc/passwd.
    (Show solution in sed)
    #!/bin/sh

    sed '10,20s/^/#/' /etc/passwd

    (Show solution in ed)
    #!/bin/sh

    ed /etc/passwd <<\END
    10,20s/^/#/
    %p
    Q
    END
  4. Using sed put character “o” to the beginning of each odd line in file /etc/passwd and “e” to the beginning of each even line in this file.
    (Show solution in sed)
    #!/usr/bin/sed -f
    # This is a sed script, use with sed -f
    # Usage: sed -f <script> /etc/passwd

    s/^/o/
    n
    s/^/e/

    (Show solution in ed)
    #!/bin/sh

    # The modified file is not written (it would not be a good idea in
    # case of /etc/passwd).

    ed /etc/passwd <<\END
    g/./s/^/o/\
    +1s/^/e/
    %p
    Q
    END
  5. Using sed insert an empty line between each two lines of /etc/passwd.
    (Show solution in sed)
    #!/bin/sh

    # The $b command ensures that nothing will be added after the last
    # line.
    # Command a performs the actual empty line append.

    sed '$b; a\
    '
    /etc/passwd

    #  Equivalently we can do it in the following way with i (the
    addresses ensure that nothing is added before the first line):

    sed '2,$i\
    '
    /etc/passwd

    (Show solution in ed)
    #!/bin/sh

    # Backslash in front of END prevents shell from performing any
    # expansions in the text before END.
    #
    # The backslash after i command is there because it is a command
    # within global g command, otherwise there would be no backslash after
    # i.

    ed /etc/passwd <<\END
    2,$g/.*/i\
    \
    .
    %p
    Q
    END
  6. Write a shell script which expects two parameters, a file name and a number. The script deletes the line with given number from given file.
    (Show solution)
    #!/bin/sh
    sed -e "${2}d" "$1" >/tmp/withoutline.$$
    cp /tmp/withoutline.$$ "$1"
    rm /tmp/withoutline.$$

    # Using cp+rm instead of mv keeps the same inode of the original file,
    # which may be useful if it has several hard links.
  7. Write a script which expects n+1 parameters, the first parameter contains a number x between 1 and n, the script then outputs the (x+1)st parameter. For instance script 2 a b c outputs b, i.e. the third parameter (or the second if we count starting from a. (Use shift).
    (Show solution)
    #!/bin/sh

    shift "$1"
    echo "$1"

Homeworks:

  1. Using sed replace in the input text file all occurrences of characters '&', '<', '>' with their HTML entities (i.e. replace '&' with '&amp;', '<' with '&lt;', and '>' with '&gt;'). Call sed just once. [1 point]
  2. Using sed join each two consecutive lines of the input to one (i.e 1+2, 3+4, ...). [1 point]
  3. Using sed print only odd lines from the output of nl /etc/passwd. [1 point]

9th practical (21st April, 2016)

Commands: read, compound commands (for, while, if, case, etc.), printf, expr, test, dirname, basename.

Other: Variables and expansion, redirection, positional and special parameters/variables ($#, $0, $n, ${n}, shift, "$@", set -, $?, $$, $!), command combination using &&, command grouping and subshell invokation, back apostrophes and $() expansion.

  1. Write a script which expects one or two number parameters. The script then pads the first number to number of digits given in the second parameter. If the second parameter is not specified, it is considered to be 5. (Use printf and expansion ${variable:-word}.)
    (Show solution)
    #!/bin/sh

    printf "%${2:-5}d\n" $1
  2. What the following lines write to the screen?
    a="hello"; { echo $a ; a="hi" ; echo $a ; } ; echo $a
    b="hello"; ( echo $b ; b="hi" ; echo $b ; ) ; echo $b
    x="hello"; x="hi" sh -c 'echo $x'; echo $x
    y="hello"; y="hi"; sh -c 'echo $y'; echo $y
    z="hello"; sh -c 'echo $z'
    export u="hello"; sh -c 'echo $u'
  3. What is the following command doing?
    a=1;b=2; { a=LEFT; echo $a >&2; } |\
    { b=RIGHT; echo $b; tr 'A-Z' 'a-z';}; echo "A=$a,B=$b"
    And how is it different from:
    a=1;b=2; { a=LEFT; echo $a; } |\
    { b=RIGHT; echo $b; tr 'A-Z' 'a-z';}; echo "A=$a,B=$b"
  4. Write a shell script which expects file paths in its parameters. The paths in parameters can be relative. For each path in parameter the script outputs the full path of the file.
    (Show solution)
    #!/bin/sh

    for x
    do
       directory=$(dirname "$x")
       directory=$(cd $directory && pwd)
       echo $directory/$(basename $x)
    done
  5. Will the following script correctly compute the number of lines in /etc/passwd?
    count=0;
    cat /etc/passwd | while read x
    do
       count=`expr $count + 1`
    done
    echo $count
    And what about the following script?
    cat /etc/passwd | {
       count=0
       while read x
       do
          count=`expr $count + 1`
       done
       echo $count
    }
    And what about the following script?
    count=0
    while read x
    do
       count=`expr $count + 1`
    done </etc/passwd
    echo $count
    And finally, what about the following script?
    count=0
    while read x </etc/passwd
    do
       count=`expr $count + 1`
    done
    echo $count
    What is the difference between all these scripts?
  6. Write a shell script which renames all files in the current directory to files with the same name but where the upper case characters are changed into lower case. (E.g. file BACKUP.ZIP would be renamed to backup.zip.)
    (Show solution)
    #!/bin/sh

    for x in *[A-Z]*
    do
       mv "$x" "`echo $x | tr '[:upper:]' '[:lower:]'`"
    done
  7. Write a shell script which renames all files in the current directory with suffix “.jpeg” to the files with the same name but with their suffix changed to “.jpg”.
    (Show solution)
    #!/bin/sh

    # A variant using special variable expansion (not for too long in
    # the standard though).
    for x in *.jpeg
    do
       mv "$x" "${x%jpeg}jpg"
    done

    # A variant using sed:
    for x in *.jpeg
    do
       mv "$x" "$(echo $x | sed 's/\.jpeg$/.jpg/')"
    done

    # A variant using basename and dirname.
    for x in *.jpeg
    do
       mv "$x" "$(dirname "$x")/$(basename "$x" .jpeg).jpg"
    done
  8. Write a shell script which receives three parameters, all of them are numbers. The script outputs a list of numbers separated with spaces. The numbers start with $1, end with $2 and they are increased with step given in $3. If the third parameter is missing, then step 1 is considered by default.
    (Show solution)
    #!/bin/sh

    if [ $# -lt 2 ] || [ $# -gt 3 ]
    then
       echo Bad number of parameters.
       exit 1
    fi

    step=${3:-1}

    x=$1
    while [ $x -le $2 ]
    do
       printf "$x "
       x=`expr $x + $step`
    done
  9. The same as the previous exercise, but now each number is on a separate line and they are justified to the number of digits of the longest number (i.e. $2). The numbers are justified with leading zeros, i.e.
    script 0 10 2
    would output:
    00
    02
    04
    06
    08
    10
    
    (Show solution)
    #!/bin/sh
    if [ $# -lt 2 ] || [ $# -gt 3 ]
    then
       echo Bad number of parameters
       exit 1
    fi

    step=${3:-1}

    x=$1
    len=${#2}
    while [ $x -le $2 ]
    do
       printf "%0${len}d\n" $x
       x=`expr $x + $step`
    done
  10. Write a shell script which renames all files in the current directory having suffix “.jpg” to files with the same suffix but with numbers instead of names. The numbers are justified to the number of digits as necessary (given by the number of files). (Use the exercises of this practical). For example if the current directory would contain files alice.jpg, bob.jpg, cyril.jpg, daniel.jpg, they would be renamed to 1.jpg, 2.jpg, 3.jpg, and 4.jpg. If there would be at least 10 but at most 99 files having suffix “.jpg”, they would get names 01.jpg, 02.jpg, 03.jpg, 04.jpg, ...
    (Show solution)
    #!/bin/sh
    # If by any chance there is a file with name with the number in the
    # required output format, then this script might cause some problems.
    # In that case it is necessary to first move the files (under the new
    # names) to a temporary directory and back.

    x=1
    count=`ls *.jpg | wc -l | tr -d " "`
    for name in *.jpg
    do
       mv "$name" `printf "%0${#count}d\n" $x`.jpg
       x=`expr $x + 1`
    done
  11. Write a shell script which reads the lines from the standard input of the form:
    a+b=c
    where a, b, and c are numbers. The script reads the numbers on each line and checks if the equality really holds. The lines not satisfying the equality are then reported to standard output. At the end a total number of incorrect lines is written to standard output.
    Hint: Use read in combination with IFS="+=".
    (Show solution)
    #!/bin/sh
    IFS="+="

    line=0

    while read a b c
    do
       line=$(expr $line + 1)
       if expr "$a" + "$b" != "$c"
       then
          printf "Error on line %d (%d+%d!=%d).\n" $line "$a" "$b" "$c"
       fi
    done
    printf "Number of processed lines: %d\n" $line

Homeworks:

  1. Write a script which works as cp but with reversed order of parameters (the target is the first parameter). [1 point]
  2. Write a script which expects three parameters, a file name, a number n and a character c. The script then performs a circular shift of each input line (input is in the file specified in the first parameter). The lines are considered to be split into fields separated with character c and the number of elements to shift is given in number n. For example if the script is called like
    skript /etc/passwd 2 :
    then each line of /etc/passwd is transformed from the initial format
    login:*:uid:gid:full name:home dir:shell
    to the following form:
    uid:gid:full name:home dir:shell:login:*
    The modified contents of the input file is written to the standard output, the file itself is left untouched. [2 points]

10th practical (28th April, 2016)

  1. Write a shell script which expect the lines of the input in the following form:
    goal : z1 z2 .... zn
    where goal, z1, z2, ..., zn are file names (paths). The script will list the goals which are older than any file of z1, z2, z3 ..., zn (subgoals) on the same line. (Note that parameter -nt of test is not standard, you can use find with parameter -newer or ls with parameter -t.)
    (Show solution)
    #!/bin/sh

       # The solution would not work if there would be spaces in file
       # names, but then we would have to specify how to distinguish a
       # space separating two file names and a space being a part of a
       # filename, so we will ignore spaces in filenames for now.

    while read goal colon files
    do
       set -- $files
       if [ "$(find "$@" -newer "$goal" -prune)" ]
       then
          echo $goal
       fi
    done

    # An alternative solution using ls -t:
    while read goal colon files
    do
       if [ "$(ls -t "$goal" $files | head -n 1)" != "$goal" ]
       then
          echo $goal
       fi
    done

Homework

  1. Write a script csvcut, which accepts two or more parameters with the following meaning: The 1st parameter is a delimiter character delim, the 2nd parameter is a string specifying a column in a CSV file, the 3rd and following parameter specify file names (if these names are missing, the script reads standard input). The script assumes that the input files are CSV files with delim as a delimiter character. Moreover the script assumes that the first line of each of these files contains a header. The script cuts the specified column from the input files, i.e. it cuts the column whose name is equal to the 2nd parameter of the script, regardless the possible quotation marks. For example
    • csvcut ';' Measure calories.csv
      cuts the second column from file calories.csv that is it would be equivalent to cut -d ';' -f 2 calories.csv).
    • csvcut ';' Food calories.csv
      cuts the first column from the same file although the name is surrounded with quotation marks.
    • [3 points]

11th practical (5th May, 2016)

Commands: trap, ps, kill, sleep, date.

  1. Write a script which prints every line it reads from the standard input. If the script receives signal KILL (9), then it naturally stops. If the script receives one of signals TERM (15), QUIT (3), INT (2) (the one after CTRL-C), the script does not stop, but it only prints out the number of singal it received. In case of INT (2) the script moreover prints out the last line which was read from the input.
    (Show solution)
    #!/bin/sh

    trap "echo 15" TERM
    trap "echo 3" QUIT
    trap 'echo 2; echo $x' INT

    while read x
    do
       echo $x
    done
  2. Write a script which expects two parameters, a signal and a program name. The script then sends the signal to all processes of the program. You can assume that the program name does not contain any weird characters (it can contain a space, however). That is, the aim is to create something like killall, but killall itself cannot be used as it is not a standard command. I suggest to look at parameter -o of command ps. You can further use e.g. sed and xargs.
    (Show solution)
    #!/bin/sh

    [ $# -eq 2 ] || {
       echo "Expecting two parameters, a signal and a program name.";
       exit 1
    }

    signal="$1"
    program="$2"

    ps -e -o pid= -o comm= |\
    sed -n 's/^ *\([[:digit:]]\{1,\}\) \{1,\}'"$program"'$/\1/p' |\
    xargs kill -$signal

    # The same can be achieved using a while cycle.
    ps -e -o pid= -o comm= | while read pid cmd
    do
       if [ "$cmd" == "$2" ]
       then
          kill -$signal $pid
       fi
    done
  3. Write a script which expects one parameter specifying number of seconds. The script then waits for given number of seconds before it finishes. When the script receives signal SIGINT (2 - Ctrl-C), then it writes down how many seconds did it wait until now. When the script receives signal SIGQUIT (3 - Ctrl-\), then it writes down how many seconds remain. When the script receives SIGTERM (15), then it writes down how many seconds did it wait, how many seconds remain and stops. The exit code of the script should be 0 if it finishes in the required time, or the remaining number seconds if it was terminated earlier (due to SIGTERM (15)).
    Do not use date +%s, because %s is not a standard part of a format string in date. Use sleep in a cycle.
    (Show solution)
    #!/bin/sh

    die() {
       echo "$1"
       exit 1;
    }

    # It is sufficient to consider hours, minutes and seconds, we assume
    # that the interval is shorter than a day and that the interval will
    # not go through midnight. (Just for simplicity.)
    now() {
       equation=$(date +"%H \* 3600 + %M \* 60 + %S")
       eval expr $equation
    }

    [ $# -eq 1 ] || die "Bad number of parameters"
    expr "$1" : '[[:digit:]]*$' >/dev/null || die "Parameter is not a number"

    start=$(now)
    finish=$(expr $start + $1)

    trap 'expr $(now) - $start' 2
    trap 'expr $finish - $(now)' 3
    trap 'expr $(now) - $start; kill $! 2>/dev/null; exit' 15

    while [ $(now) -lt $finish ]
    do
       sleep $(expr $finish - $(now) ) &
       wait $!
       [ $? -gt 128 ] && kill $! 2>/dev/null
    done

    # It would be sufficient to use sleep 1 in the above cycle, but then
    # the process would wake up each second.
    # sleep $(expr $konec - $(now) ) is not enough by itself because of SIGTERM,
    # if the shell process running the script receives SIGTERM when sleep
    # is running, then it first lets sleep to finish and then it processes
    # the signal using trap. (This is not the case of Ctrl-C as there the
    # signal is received by sleep.) That is why sleep is being run on
    # background and then it is waited for the child process to finish.
    # After wait the signal is processed and SIGTERM is passed to the
    # sleep process if necessary. Wait returns 128 if it was terminated
    # due to a signal.
  4. Using sed reverse the order of lines in the input.
    (Show solution in sed)
    #!/bin/sh

    # We shall try it with nl /etc/passwd to see the difference.
    nl /etc/passwd | sed -n 'G; $p; h'
  5. If a line in the input text file ends with a backslash, it means it continues on the next line. Write a sed script which joins the lines if the first of them ends with a backslash (which is replaced with a space). For instance consider a text file:
    a b c\
    d e f\
    g h i
    j k l\
    m n o
    
    Then the script produces
    a b c d e f g h i
    j k l m n o
    

    (Show solution in sed)
    #!/bin/sed -f
    : a
    /\\$/ N
    s/\\\n */ /
    t a

Homeworks:

  1. Assume that the input lines contain only characters '(' and ')', write a sed script which checks whether each input line contains string of well paired brackets. The correct lines are replaced with 'ok' string, the wrong lines with 'wrong' string. (e.g. string '(())()((()))' would be 'ok', while string '(((())()' would be 'wrong') [1 point]
  2. At the input assume a text file in which paragraphs are separated by an empty line. Write a sed script which joins each paragraph to one line and removes the empty lines. [2 points]

12th practical (12th May, 2016)

Command: awk.

  1. Write an awk script which outputs each tenth line of the input.
    (Show solution)
    #!/bin/awk -f

    (NR % 10) == 0
  2. Write an awk script which changes the characters in login field of /etc/passwd file to upper case.
    (Show solution)
    #!/bin/awk -f

    BEGIN { FS=":"; OFS=":" }
    { $1=toupper($1); print }
  3. Write an awk script, which reverses the order of words at each line of the input (writes the words in opposite order).
    (Show solution)
    #!/bin/awk -f

    {
       for (pole = NF; pole > 1; pole --)
       {
          printf ("%s" OFS, $pole);
       }
       printf ("%s" ORS, $1);
    }
  4. Write an awk script which numbers the lines of the input (similarly to nl).
    (Show solution)
    #!/bin/awk -f

    {
       print NR " " $0;
    }
  5. Write an awk script which lists random numbers between 0 and 1, the number of random number to be output is given in the parameter.
    (Show solution)
    #!/bin/sh
    # This is a shell script which encapsulates the awk script.

    awk -v pocet="$2" 'BEGIN {
       srand();
       for (i=0; i<pocet; ++i)
       {
          print rand ();
       }
    }'
  6. Write an awk script which computes the number of HTML links in an HTML file (i.e. counts the tags <a).
    (Show solution)
    #!/bin/awk -f
    # RS will be "<", because that is how HTML is separated.

    BEGIN {
       RS = "<";
       cnt = 0;
    }

    ($1 ~ /^a$/) {
       ++ cnt;
    }

    END {
       print cnt;
    }
  7. Write an awk script which expects an HTML file at the input and lists the web addresses which the file references (using tag <a href="address">. Note that the tag can be split over several lines and that there can be spaces around a and href.
    Hint: Use suitable RS and FS values.
    (Show solution)
    #!/bin/awk -f
    # The solution is a bit complicated so that it works even if there are
    # other parameters to a between a and href. Otherwise it would be
    # enough to take i=1.

    BEGIN {
       RS = "<";
       FS = "=";
    }

    $1 ~ /^[[:blank:]]*a[[:blank:]]+/ {
       i=1
       while (i <= NF)
       {
          if ($i ~ /[[:blank:]]+href[[:blank:]]*$/)
          {
             split ($(i+1), a, "[[:blank:]]*\"[[:blank:]]*");
             print a[2]
          }
          else if ($i ~ />/)
          {
             break;
          }
          ++i;
       }
    }
  8. Write an awk scripts which outputs the last 10 lines of the input file.
    (Show solution)
    #!/bin/awk -f

    BEGIN {
       konecBufferu = 0;
    }

    {
       buffer [konecBufferu] = $0;
       konecBufferu = (konecBufferu + 1) % 10;
    }

    END {
       if (NR > 0)
       {
          if (NR < 10)
          {
             indexVBufferu = 0;
          }
          else
          {
             indexVBufferu = konecBufferu;
          }
          do
          {
             print buffer[indexVBufferu];
             indexVBufferu = (indexVBufferu + 1) % 10;
          }
          while (indexVBufferu != konecBufferu)
       }
    }

Homeworks:

  1. Write an awk script which expects a file in the csv format at the input. This means that each line of the input line consists of several fields separated with a comma ",". The first line is the header line, it contains the column names. The first column of the subsequent lines contains a name, the rest of columns contain numbers. The script adds a new column called “Sum” (in the header line), which on each line contains the sum of the number fields. For example in case of input
    Name,pts1,pts2,pts3
    Hynek,3,12,9
    Jarmila,7,34,1
    Vilém,8,27,0
    
    the following output is produced by the awk script:
    Name,pts1,pts2,pts3,Sum
    Hynek,3,12,9,24
    Jarmila,7,34,1,42
    Vilém,8,27,0,35
    
    [3 points]

13th seminar (19th May, 2016)

  1. Write a script which receives four parameters, dir, mask, regexp, and repl. The script then replaces all occurences of regular expression regexp with replacement string repl in every file whose name matches shell mask mask in the subtree of directory given by path dir.
    For instance script src '*.c' 'var1' 'var2' replaces all occurences of var1 with var2 in all files with extension .c in the subtree of directory src (within the current working directory).
    (Show solution)
    #!/bin/sh

    if [ $# -ne 4 ]
    then
       echo Too few parameters, expected 4.
       exit 1
    fi

    dir="${1:-.}"
    mask="${2:-*}"
    regexp="$3"
    repl="$4"

    find "$dir" -name "$mask" -type f | while read -r fn
    do
       ed "$fn" <<END
    g/./s/$regexp/$repl/g
    w
    q
    END

    done
  2. Write a script newfrom which receives pathes to two directories DIR1 and DIR2. The files in DIR1 which either are not in DIR2, or they are newer in DIR1 than in DIR2 are copied from DIR1 to DIR2. The script works recursively also on the subdirectories of DIR1 and DIR2. The script accepts an optional switch -n, if this switch is provided, the script does not actually copy anything, just lists the files which would be copied had -n switch not been provided. In case DIR2 does not exist, it is created. The same is true for subdirectories.
    (Show solution)
    #!/bin/sh

    usage() {
       echo "Usage: skript [-n] dir1 dir2"
       exit 1
    }

    if [ "(" $# -eq 3 ")" -a "(" "$1" == "-n" ")" ]
    then
       DRY_RUN=1
       shift
    else
       DRY_RUN=0
    fi

    if [ $# -ne 2 ]
    then
       usage
    fi

    DIR1="$1"
    DIR2="$2"

    if [ ! -d "$DIR1" ]
    then
       echo "$DIR1 is not a directory"
       usage
    fi

    if [ ! -d "$DIR2" ]
    then
       if [ $DRY_RUN -eq 0 ]
       then
          cp -pR "$DIR1" "$DIR2"
       else
          echo cp -pR "$DIR1" "$DIR2"
       fi
    else
       ( cd "$DIR1" && find . -type d -path "*/*" ) >/tmp/tempfile1.$$
       while read line
       do
          rel_path="`echo "$line" | cut -d / -f 2- `"
          if [ ! -d "$DIR2/$rel_path" ]
          then
             if [ $DRY_RUN -eq 0 ]
             then
                mkdir "$DIR2/$rel_path"
             else
                echo "mkdir $DIR2/$rel_path"
             fi
          fi
       done </tmp/tempfile1.$$

       ( cd "$DIR1" && find . -type f -path "*/*" ) >/tmp/tempfile1.$$
       while read line
       do
          rel_path="`echo "$line" | cut -d / -f 2- `"
          if [ "`find "$DIR1/$rel_path" -newer "$DIR2/$rel_path"`" ]
          then
             if [ $DRY_RUN -eq 0 ]
             then
                cp -p "$DIR1/$rel_path" "$DIR2/$rel_path"
             else
                echo "cp -p $DIR1/$rel_path $DIR2/$rel_path"
             fi
          fi
       done </tmp/tempfile1.$$
    fi

    rm /tmp/tempfile1.$$

Homework:

  1. Write a script rmexcept, which expects n+1 parameters where the first parameter is a directory and the rest of parameters contain shell masks specifying file names. The script removes all files whose names do not match any of the given shell masks from the given directory. The files whose names match one of the masks, are left untouched (in particular they cannot be moved anywhere else and then back). E.g. calling rmexcept . '*.jpg' '*.png'
    would remove all files whose suffix is neither jpg nor png from the current working directory. [3 points]

14th practical (26th May, 2016)

  1. Write a script which replaces all C style comments (i.e. /* */) to C++ style comments (i.e. //). Be aware of the following facts:
    • C style comments can be on a single line, but they can be multi-line as well.
    • If there is something after a C style comment, you have to move it to the next line.
    • There can be multiple C style comments on a single line.
    You may assume that the input file does not contain other occurrences of /* and */ except those which delimit a comment, that the comments are not nested and that the comment delimiters are well paired. For example
    aaa /* bbb */ ccc /* ddd */ eee /* fff
    ggg */ hhh /* iii */ jjj
    kkk
    /* lll
       mmm
       nnn */
    ooo
    
    would be turned into
    aaa // bbb
    ccc // ddd
    eee // fff
    // ggg
    hhh // iii
    jjj
    kkk
    // lll
    // mmm
    // nnn
    ooo
    
    The output of your script may include additinal spaces (or newlines), but the output should be valid.
  2. Write a script which receives number of seconds sec, a program name prg followed by its arguments. The script runs program prg with its arguments and then waits for it to finish. If the program does not finish within sec seconds, the script sends signal TERM to prg. If after at most another 5 seconds the program still does not finished, the script kills it by sending signal KILL to it. For simplicity you may assume that the interval given by number of seconds does not go accross a midnight to the next day.

Additional homeworks (if you do not have enough points).

  1. Write a sed script which reads a decimal number n from the input and writes sequence 0, 1, ..., n to the output. Each number is on a separate line. [2 point]
  2. Write a shell script which selects n longest lines from the files passed to it in parameters. Each output line is prepended with its number within the file (format line number: line content). The lines in the output are in the same order as they are in the input file, i.e. as if sorted by their line number. Parameter n is 5 by default but it can be specified in the first parameter as -n10 or in the first two parameters as -n 10 (this is for the case when n=10). The syntax is thus same as in case of head. If the file names are missing in parameters, standard input is used. If at least two files are specified in parameters, each list of lines is preceded by the corresponding file name. [3 points]
  3. Write a script which will indent lines in the input files depending on how deep the brackets are nested. The parameters to the script are an indentation character c and a number of characters per level n, these parameters are followed by a list of files (if no files are provided, standard input is used). The script then reads line by line and on each line it removes the white characters at the beginning of line and replaces them with k*n characters c where k is the level of bracket nesting. Consider normal brackets (), curly brackets {} and square brackets []. For example the input file
    a ( b
         c d [ e ] f [
      g h { j (
                k ) } l m
         n ] o ) p
    q r
    
    would be modified as follows if the script is run with parameters c='.' and n=1:
    a ( b
    .c d [ e ] f [
    ..g h { j (
    ....k ) } l m
    ..n ] o ) p
    q r
    
    You can assume that the input has well paired brackets. A space or a tab character must be allowed as character c. [3 points]