# Basic information

• Teacher: Petr Kučera <>, room S304
• Schedule: Wednesday 12:20 — 13:50 SU1

# Credit requirements

At the end of each practical homework shall be assigned, I will expect to receive solutions to them by e-mail before the beginning of the next practical. If by the end of the semester you have at least 2/3 of possible number of points, you get a credit. Otherwise you will need to finish additional homework to get enough points. In case you submit solutions to less than 1/3 of homework assignments, you will not have a possibility to get a credit. Each set of homework exercises assigned after each practical will be worth 3 points, at the beginning of the semester that could mean three exercises while at the end it can be just a single but more difficult exercise.

If not stated otherwise, you are only allowed to use the commands, tools and structures we have used on practicals before the homework was assigned (including the one to which homework was assigned). In some cases the tools might be even more restricted. If I come across a solution which is just a copy of another one I will assign no points for this solution. If this case repeats there will be no possibility to obtain a credit from me.

Please, send me your solutions via an e-mail and do so before the beginning of the next practical. Either write your solution as a part of e-mail body, or append them in a form of plain text files, do not use pdf, word or similar file types. The homework assignments are appended below a list of exercises we have done at the corresponding practical (i.e. the current homework exercises are always at the bottom of this web page).

There have been 13 homework assignments and thus the number of points required to get the credit is 26. Additional homework exercises for those who do note have enough points will be added to the last exercise.

# Some peculiarities of the Malá Strana UNIX lab

I think it is worth to know and bear in mind some specifics of the Malá Strana UNIX lab. To Czech students I recommend to check the web pages of the UNIX lab. I will warn you of the lab properties as we come across them. In this section I would to mention at least the most apparent of them:

1. Users home directories are placed in a remote AFS filesystem, where standard UNIX permissions do not apply and the standard UNIX commands for file permissions manipulations (mainly chmod) cannot be used or they can be used only in a very restricted way (there is a web page about the ACL which is the system in use, but it is only in Czech to the best of my knowledge).
2. This means that when you try the chmod command or anything which has some connection to the standard UNIX permission system, you should do so in local directories (such as /tmp in which you are allowed more or less anything). There the standard UNIX permissions apply.
3. As user accounts are stored remotely, local file /etc/passwd does not contain information about your account or accounts of other lab users. If you want to get your and other users account info in the format of /etc/passwd you can achieve that using a getent command as follows:
getent passwd
for other parameters and general help on this command use getent --help.
4. The same applies to /etc/group and getent group.
5. The find command (still most probably) is not in the best of terms with AFS filesystem — when used on network AFS drives in the lab such as your home dir you can get unexpected results. You can use find to search on local directories such as /tmp, /etc, /usr etc. The exercises usually suggest it.

# Content of the practicals

## 1st practical (26th February, 2016)

Basic commands, shell command line edit.

More detailed:

• Command line edit: Ctrl-a, Ctrl-e, Ctrl-h, Ctrl-r, arrows (left, right, up and down), Ctrl-d, Ctrl-k (deletes text from the cursor position to the end of line).
• Commands: man, ls, cd, pwd, cat, more, less, head, tail, wc, mkdir, rmdir, cp, mv, rm, echo.
• Redirecting and pipes: <, >, >>, 2>, 2>>, |, /dev/null.
• Shell expansion: *, ?, [list], [!list], [[:class:]], apostrophes and quotation marks, filenames starting with a dot.

### Exercises

1. In your home directory create a directory named DIR
2. Copy all files whose filenames satisfy the following conditions to ~/DIR. The files are in /usr/include directory, their names start with m, end with .h and contain a number.
3. Create a subdirectory called SUBDIR in your DIR directory.
4. The first five lines of each file you have copied from /usr/include copy to file ~/DIR/SUBDIR/firstfive.
5. The last lines of files in ~/DIR copy to file ~/DIR/SUBDIR/last.
6. Concatenate the two files in ~/DIR/SUBDIR into one file ~/DIR/SUBDIR/firstandlast
7. Delete the files in ~/DIR/SUBDIR except firstandlast.
8. Store the number of files and directories in ~/DIR into a file ~/DIR/SUBDIR/count
9. Output the long information on ~/DIR/SUBDIR directory. (Not its content, but information on it).
10. Delete the contents of ~/DIR/SUBDIR/firstandlast file without removing the file itself.
11. Add a line containing just a star sign (i.e. *) to file ~/DIR/SUBDIR/firstandlast.
12. Delete ~/DIR together with all the files it contains.
13. (Show solutions to preceding exercises)
#!/bin/sh # In your home directory create a directory named DIR mkdir ~/DIR # Copy all files whose filenames satisfy the following conditions to # ~/DIR. The files are in /usr/include directory, # their names start with 'm', end with '.h' and contain a # number. cp /usr/include/m*[0-9]*.h ~/DIR # Create a subdirectory called SUBDIR in your DIR directory. mkdir ~/DIR/SUBDIR # The first five lines of each file you have copied from # /usr/include copy to file ~/DIR/SUBDIR/firstfive. head -n 5 ~/DIR/m*[0-9]*.h >~/DIR/SUBDIR/firstfive # The last lines of files in ~/DIR copy to file # ~/DIR/SUBDIR/last. tail -n 1 ~/DIR/m*[0-9]*.h >~/DIR/SUBDIR/last # Concatenate the two files in ~/DIR/SUBDIR into one file # ~/DIR/SUBDIR/firstandlast cat ~/DIR/SUBDIR/* >~/DIR/SUBDIR/firstandlast # Delete the files in ~/DIR/SUBDIR except firstandlast. rm ~/DIR/SUBDIR/firstfive rm ~/DIR/SUBDIR/last # Store the number of files and directories in ~/DIR into a # file ~/DIR/SUBDIR/count ls ~/DIR | wc -l >~/DIR/SUBDIR/count # Output the long information about ~/DIR/SUBDIR directory. # (Not its content, but information about it). ls -ld ~/DIR/SUBDIR # Delete the contents of ~/DIR/SUBDIR/firstandlast file # without removing the file itself. cp /dev/null ~/DIR/SUBDIR/firstandlast # Add a line containing just a star sign (i.e. *) to file # ~/DIR/SUBDIR/firstandlast. echo '*' >>~/DIR/SUBDIR/firstandlast # Delete ~/DIR together with all files the it contains. rm -r ~/DIR
14. Output lines number 11-20 from file /etc/passwd.
(Show solution)
#!/bin/sh head -n20 /etc/passwd | tail -n10

### Homework

1. Write a command which lists all files in your home directory whose name starts with a dot (“.”) and contain at least two other characters. [1 point]
2. Write a command which prints the last line of each file in directory /usr/include whose names end with suffix “.h”. [1 point]
3. Create a directory INCL in your home directory. Write a command which copies files whose names do not start with a digit but contain a digit in the name from directory /usr/include to the above created directory INCL in your home directory. [1 point]

## 2nd practical (28th February, 2018)

Basic commands and shell usage.

Commands: date, tee, touch, cut, paste, tr.

### Exercises

1. Create a file with name “-f” which contains a line with current date and time. Then delete this file.
(Show solution)
#!/bin/sh # Create a file -f with date and time: date >-f # Delete file -f: rm -- -f
2. Create a file named “fixed date” (i.e. “fixed<space>date”) with last date and time of modification is set to 13:30, 1st February, 2009.
(Show solution)
#!/bin/sh touch -t 200902011330 "fixed date" # or touch -d "2009-02-01 13:30:00" "fixed date"
3. Copy the file /etc/passwd into your home directory. The copy should have name accounts and the last modification date and time should be preserved, i.e. should be the same as the last modification date and time of /etc/passwd.
(Show solution)
#!/bin/sh cp -p /etc/passwd ucty
4. Set the time and date of last modification of file accounts to the same time as the last modification time of the file /etc/group.
(Show solution)
#!/bin/sh touch -r /etc/group accounts
5. Write a command which outputs the list of files in directory /usr/bin sorted lexicographically in decreasing order. The output is written to files bina, binb, and to the screen. The screen output is viewed in pages (use less, or more).
(Show solution)
#!/bin/sh ls -r /usr/bin | tee bina binb | more
6. What happens during the execution of the following three commands? What is the difference betweeen them?
1. mv file /dev/null
2. cp file /dev/null
3. cat file >/dev/null
7. Write commands that output the ten biggest and the ten smallest files in directory /etc.
(Show solution)
#!/bin/sh ls -S /etc | head ls -rS /etc | head
8. Find a file in /usr/bin with the most recent modification date and time.
(Show solution)
#!/bin/sh ls -t /usr/bin | head -n 1
9. Download file calories.csv, it is a file in a CSV format with semicolon (“;”) used as a field separator.
10. Replace quotation marks (") with apostrophes (') in file calories.csv
(Show solution)
#!/bin/sh tr \" \' <calories.csv >/tmp/calories.a.csv mv /tmp/calories.a.csv calories.csv
11. Remove apostrophes (') from file calories.csv. In this and the several following exercises we assume that the file is has been modified by preceding exercises, i.e. here after the previous exercise, quotation marks have been replaced with apostrophes.
(Show solution)
#!/bin/sh tr -d \' <calories.csv >/tmp/calories.a.csv mv /tmp/calories.a.csv calories.csv
12. Select the first column from file calories.csv, i.e. the column with food names.
(Show solution)
#!/bin/sh cut -f1 -d\; calories.csv
13. In file calories.csv replace upper case characters with their lower case counterparts, but only in the first column. (Cut the first columns using cut, replace capital letters with small ones using tr, then put them back using paste.)
(Show solution)
#!/bin/sh cut -f1 -d\; calories.csv >/tmp/first cut -f2- -d\; calories.csv >/tmp/others tr '[:upper:]' '[:lower:]' </tmp/first | paste -d\; - /tmp/others >calories.csv rm /tmp/first /tmp/others
14. Reorder the last three columns of file calories.csv in the opposite order, i.e. after reordering the first column contains amount of proteins, the second contains amount of carbohydrates and the third column contains the amount of fat. Use cut and paste.
(Show solution)
#!/bin/sh cut -f5 -d\; calories.csv >/tmp/fifth cut -f6 -d\; calories.csv >/tmp/sixth cut -f7 -d\; calories.csv >/tmp/seventh cut -f1-4 -d\; calories.csv | paste -d\; - /tmp/seventh /tmp/sixth /tmp/fifth >/tmp/novy mv /tmp/novy calories.csv rm /tmp/fifth /tmp/sixth /tmp/seventh
15. Select the 9 characters with permissions from the output of ls -l command.
(Show solution)
#!/bin/sh ls -l | tail -n +2 | cut -c2-10
16. Suppose you have three files, summand1, summand2 and sum, each of these files contains the same number of lines, each line contains just a number. Compose these files to just one output where each line has the following format:
summand1+summand2=sum.
(Show solution)
#!/bin/sh paste -d+= summand1 summand2 sum

### Homework

1. Write a command which sets the last modification date and time of a file to 11th March, 1993, 15:04. [1 point]
2. In file calories.csv replace each food name with a “-” (minus) character. Other columns should remain unchanged. [1 point]
3. For each user in /etc/passwd output a pair uid:login. [1 point].

## 3rd practical (7th March, 2018)

Permissions, links, communication utilities.

Commands: ssh, scp, who, id, mail (mailx), last (not a standard command), chmod, chgrp, chown, file, ln, df, du, umask.

Attention, in Malá Strana UNIX lab try permissions in /tmp directory. Other directories are on afs file system which has a different system of permission handling.

### Exercises

1. Output the list of files in the current directory in a form of “size filename”. The list should be created by filtering the outputs of ls -l and ls commands.
(Show solution)
#!/bin/sh ls -l | tail -n +2 | tr -s " " | cut -d" " -f5 >/tmp/lsl ls | paste -d" " /tmp/lsl - rm /tmp/lsl
2. Output the logins from file /etc/passwd, each line contains 5 logins separated with a comma (,).
(Show solution)
#!/bin/sh cut -d: -f1 </etc/passwd | paste -d"," - - - - - # Alternatively and equivalently: cut -d: -f1 </etc/passwd | paste -s -d",,,,\n" -
3. Write a command which outputs your current UID.
(Show solution)
#!/bin/sh id -u
4. Write a command that outputs the list of names of groups to which you belong as a user.
(Show solution)
#!/bin/sh id -Gn
5. Send the contents of file /etc/group by an e-mail using command mail (or mailx).
(Show solution)
#!/bin/sh mail -s "Obsah /etc/group" adresa </etc/group
6. Try the command df which determines amount of space used on a disk. Try different options (such as -k and -P, you can also try the other ones but note that only other standard parameter is -t).
7. Try the command du on directory /tmp or on your home directory with different options (-k, -s, -x), find out more about these options in the specification.
8. In your home directory create a link to /etc/passwd. First try to create a hard link, then a soft link. Why a hard link cannot be created in lab? Call ls -l to find out how a soft link is displayed in its output.
(Show solution)
#!/bin/sh # Hard link ln /etc/passwd ~ # Soft link ln -s /etc/passwd ~
9. In directory /tmp create a hard link to another file in /tmp (you can copy some file to /tmp for this purpose, the hard link will naturally have a different name). Run ls -l on both files and observe how a number in the second column changed. Using ls -i check that both copies (the original file and its hard link) have the same inode number.
10. Create a simple script and assign it an execute permission.
(Show solution)
#!/bin/sh cat >script.sh <<EOF #!/bin/sh echo Hello world! EOF chmod a+x script.sh
11. Create a file which the owner cannot neither execute, read, or write to it, the members of file group can read and write to the file, and the others can just execute the and not read nor write to it.
(Show solution)
#!/bin/sh cat >script.sh <<EOF #!/bin/sh echo Hello world! EOF chmod 0761 /tmp/script.sh # Equivalently: chmod u+rwx,g=u-x,o=u-rw /tmp/script.sh # or chmod u=rwx,g=rw,o=x /tmp/script.sh
12. Create a script which cannot be read but has execute permissions. Try tu run this script. Why it is not possible to execute an unreadable shell script?
(Show solution)
#!/bin/sh cat >/tmp/script.sh <<EOF #!/bin/sh echo Hello world EOF chmod a+x-r /tmp/script.sh /tmp/script.sh # If a shell should run the script, it must be allowed to read it, too.
13. Create a directory to which anyone can go (by cd, can be used in path) and add files but only the directory owner can list its contents.
(Show solution)
#!/bin/sh mkdir /tmp/directory chmod 0755 /tmp/directory # Or equivalently chmod u+rwx,go=u-r /tmp/directory
14. Create a directory to which you can go, you cannot read its contents but you can add files to it. Create a file in this directory which can be only read (and not written to or executed). Delete this file using rm. Why you can delete a file which you cannot modify?
(Show solution)
#!/bin/sh mkdir /tmp/directory chmod 0300 /tmp/directory # Or equivalently chmod u+wx-r,go-rwx /tmp/directory touch /tmp/directory/file chmod 0400 /tmp/directory/file # Or equivalently chmod u+r-wx,go-rwx /tmp/directory/file rm -f /tmp/directory/file
15. Make a copy of /etc/passwd in /tmp using cp without and then with -p option. Observe the difference (with the help of ls -l).

### Homework

1. Write a command, which adds a group read permission to all files in a subtree of the directory /tmp/dir. Use only one call to chmod [1 point]
2. Create a file and set its permissions so that the owner can execute this file, read and write to it, group members can execute and read it, and the others can only read the file. In chmod command use a symbolic way of specifying the permissions [1 point] and also the octal numeric way [1 point].

## 4th practical (14th March, 2018)

Simple utilities.

Commands: sort, diff, comm, split.

### Exercises

1. Write the names of files which occur in /usr/bin but which are not in /bin.
(Show solution)
#!/bin/sh ls /usr/bin | sort >/tmp/usr_bin ls /bin | sort | comm -13 - /tmp/usr_bin rm /tmp/usr_bin
2. Write the numbers of groups which are used in /etc/group but which are not used in /etc/passwd. That is find the numbers of groups which are not used as a primary group for any user.
(Show solution)
#!/bin/sh cut -d: -f4 </etc/passwd | sort > /tmp/uid cut -d: -f3 </etc/group | sort | comm -23 - /tmp/uid rm /tmp/uid
3. In file calories.csv replace commas (,) with hyphens (-), using diff then determine which lines have been changed.
(Show solution)
#!/bin/sh tr "," "-" <calories.csv | diff calories.csv -
4. Sort file calories.csv according to amount of calories in increasing order. The first header line has to remain at the beginning of the file.
(Show solution)
#!/bin/sh # Save the header head -n 1 calories.csv >/tmp/header # Sort the rest and append it to the header tail -n +2 calories.csv | sort -t\; -k4,4n >>/tmp/header mv /tmp/header calories.csv
5. Sort file calories.csv according to triple [protein, carbohydrates, fat] in an increasing order. The first header line has to remain at the beginning of the file.
(Show solution)
#!/bin/sh # Save the header head -n 1 calories.csv >/tmp/header # Sort the rest and append it to the header tail -n +2 calories.csv | sort -t\; -k7,7n -k6,6n -k5,5n >>/tmp/header mv /tmp/header calories.csv
6. Determine the number of different units used in the second column of file calories.csv. First consider two units with different number as different, then as the same, such as in case of “1 Cup” and “2 cup”.
(Show solution)
#!/bin/sh # In the first solution we consider two units with a different amount # as different, e.g. “1 Cup” and “2 Cup” are different units. tail -n +2 calories.csv | sort -t\; -k2,2 -u | wc -l # If we wish to consider two units with different amounts as the same # unit (i.e. we do not want to differentiate between “1 Cup” and “2 # Cup”, we can proceed in the following way: tail -n +2 calories.csv | cut -f2 -d\; | cut -f2- -d" " | sort -u | wc -l
7. It is known that a welfare crook lives in Los Angeles. You are given lists for 1) people receiving welfare, 2) Hollywood actors, and 3) residents of Beverly Hills. The lists are given as text files where each line contains one name of a person. A welfare crook is someone who appears in all three lists. Write a script to find at least one crook. Example files: social.txt, actor.txt, beverly_hills.txt.
(Show solution)
#!/bin/sh sort social.txt >/tmp/s_social.txt sort beverly_hills.txt >/tmp/s_beverly_hills.txt sort actor.txt | comm -12 /tmp/s_social.txt - | comm -12 - /tmp/s_beverly_hills.txt rm /tmp/s_social.txt /tmp/s_beverly_hills.txt.
8. Split a file (e.g. /etc/passwd) to parts consisting of five lines, then concatenate these parts back to one file.
(Show solution)
#!/bin/sh # We shall assume that the number of lines in /etc/passwd is not too # big, in particular that when using split the two letter suffix is # enough. split -l5 /etc/passwd passwdpart # The concatenated file will be stored in the current directory, we # naturally do not have the write permission to /etc directory. cat passwdpart* >passwd rm passwdpart*
9. Write the logins from /etc/passwd to ten lines, the logins on each line are separated with spaces.
(Show solution)
#!/bin/sh # We assume that /etc/passwd does not contain too much records and # that the default two letter suffix is enough in split. cut -d":" -f1 /etc/passwd | split -l10 - logins paste -d" " logins* rm logins*
10. Create a copy of /etc/group file in which each login listed in the field with group members is replaced with a single character '@' (you may assume that a login consists of alphanumeric symbols and '_'). The other columns remain unchanged.
(Show solution)
#/bin/sh cut -d: -f4 /etc/group | tr -sc ',\n' '[@*]' >/tmp/memb.$$cut -d: -f-3 /etc/group | paste -d: - /tmp/memb.$$ >group rm /tmp/memb.$$ 11. Determine the name of a group with the highest number of members assigned in the file /etc/group. This is a follow up to the previous exercise. In particular you may assume that you already have the file produced in the previous exercise. (Show solution) #/bin/sh sort -t: -k4,4r group | head -n 1 | cut -d: -f1 ### Homework 1. Determine the login of the user with the highest uid in /etc/passwd. [1 point] 2. Determine the number of different countries which appear in file ip-by-country.csv. [1 point] 3. Based on files countrycodes_en.csv and kodyzemi_cz.csv list the names of states which are the same in English and in Czech. [1 point] ## 5th practical (21th March, 2018) Commands: find, xargs. Exercises: 1. Write a command which lists all files and directories in the subtree of current directory, whose names contain substring 'bin'. (Show solution) #!/bin/sh find . -name "*bin*" "(" -type d -o -type f ")" 2. Write a command which lists the number of directories in the subtree of /etc directory. (Show solution) #!/bin/sh find /etc -type d | wc -l 3. List all symbolic (or soft) links in a directory /usr. List only links directly present in the directory, not deeper in its subtree. (Show solution) #!/bin/sh # The following can be quite slow find /usr -type l '!' -path "/usr/*/*" # or faster find /usr/* -prune -type l # or even better find /usr $$-path "/usr/*/*" -prune$$ -o $$-type l -print$$ 4. List the files with at least three hard links pointing to them in the subtree in directory /usr/bin. (Show solution) #!/bin/sh find /usr/bin -links +2 5. Find all files in subtree of directory /tmp, which have size bigger than 100kB and which are readable by anyone. (Show solution) #!/bin/sh find /tmp -type f -size +200 -perm -a+r 6. In the subtree of /usr/include find the files with which are in depth at least 2 and at most 3 in the subtree. (Show solution) #!/bin/sh find /usr/include -path "/usr/include/*/*" ! -path "/usr/include/*/*/*/*" # A more effective solution with prune: find /usr/include -path "/usr/*/*/*" -prune -o -path "/usr/*/*" 7. In the subtree of directory /etc find all files which are newer than /etc/passwd. (Show solution) #!/bin/sh find /etc -newer /etc/passwd 8. In the subtree of directory /bin find all files which are owned by root and which can be executed by the owner and group members, but not by others. (Show solution) #!/bin/sh find /bin -user root -perm -ug+x ! -perm -o+x 9. Find all files in directory /etc which belong to group stunnel and write long information using ls -l on each of these files. (Show solution) #!/bin/sh find /etc -group stunnel -exec ls -l {} + 10. Try the difference between echo PATH; echo "PATH"; echo 'PATH'; 11. Try the difference between echo *; echo "*"; echo '*'; 12. List the paths contained in variable PATH so that each of them will be written on a separate line and there will be no separators “:”. (Show solution) #!/bin/sh echo "PATH" | tr ":" "\n" 13. Output the long information on all directories contained in variable PATH (not their contents, just the long info). (Show solution) #!/bin/sh echo "PATH" | tr ":" "\n" | xargs -I{} ls -ld {} 14. Create a file with the following three lines: a b c d e f g h i  and try to run the following commands on this file (here named “file”): cat file | xargs echo cat file | xargs echo - cat file | xargs -n 2 echo - cat file | xargs -I{} echo "-{}-" "-{}-" cat file | xargs -I{} -n 2 echo "-{}-" "-{}-"  Observe what happens when different options are passed to xargs. 15. Split the file /etc/passwd to parts consisting of five lines each and then reorder these parts in the opposite order (that is assuming /etc/passwd has n lines where n is divisible by five, the lines are now ordered as: n-4, n-3, n-2, n-1, n, n-9, n-8, n-7, n-6, n-5, ..., 1, 2, 3, 4, 5). (Show solution) #!/bin/sh split -l5 /etc/passwd passwdparts ls -r passwdparts* | xargs cat >passwd rm passwdparts* 16. Find out which directories stored in variable PATH contain a program named awk. (Show solution) #!/bin/sh echo "PATH" | tr ':' '\n' | xargs -I{} find {}/awk -prune 2>/dev/null Homework: 1. Find all files in the subtree of /etc which are not newer than /etc/group. [1 point] 2. Find the files in the subtree of current working directory which have size 0B. [1 point] 3. Find the files in the subtree of /usr/include the files whose name start with “std” but do not end with “.h”. [1 point] ## 6th practical (28th March, 2018) Commands: join, grep, sed. 1. Write commands which print a list of pairs login:group, where login is the login of a user and group is the name of primary group of user with given login. To produce this list, use join on files /etc/passwd and /etc/group (which are preprocessed as necessary). (Show solution) #!/bin/sh sort -t: -k3,3 /etc/group >skupiny sort -t: -k4,4 /etc/passwd | join -1 3 -2 4 -t: -o 2.1,1.1 skupiny - rm skupiny 2. Download files countrycodes_en.csv and kodyzemi_cz.csv. Both of these files are in a CSV format with semicolon (;) used as a separator. 3. Use files countrycodes_en.csv and kodyzemi_cz.csv to create a list of countries with two columns separated with an equality character (=) where each line has the form of Czech name=English name. (Show solution) #!/bin/sh tr -d '"' <kodyzemi_cz.csv | tail -n +2 | sort -t \; -k1,1 >/tmp/kody_s tr -d '"' <countrycodes_en.csv | tail -n +2 | sort -t \; -k4,4 >/tmp/codes_s join -t\; -1 1 -2 4 -o 1.4,2.1 /tmp/kody_s /tmp/codes_s | tr \; = >translation rm /tmp/kody_s /tmp/codes_s 4. From file calories.csv select lines in which the food name contains a character which is not alphanumeric (i.e. does not belong to [:alnum:]). (Show solution) #!/bin/sh grep '^"[^;]*[^[:alnum:];][^;]*";' calories.csv # Případně grep -v '^"[[:alnum:]]*";' calories.csv 5. From file calories.csv select lines in which amount is measured in units “Piece” or “Cake”. (Show solution) #!/bin/sh grep -e '^[^;]*;[[:digit:].]* Piece;' \ -e '^[^;]*;[[:digit:].]* Cake;' calories.csv # Případně grep -E '^[^;]*;[[:digit:].]* (Piece|Cake);' calories.csv 6. Find files in the subtree of /usr/include whose names have length 5-7 characters not including suffix which is required to by “.h”. (Use grep on the output of find command.) (Show solution) #!/bin/sh find /usr/include -name "*.h" | grep '/[^/]\{5,7\}\.h' 7. Select lines from the output of ls -l in which owner has the same permissions as group members and as others. (Create a file satisfying these condition in say /tmp if necessary.) (Show solution) #!/bin/sh ls -l | grep '^.$$...$$\1\1' 8. Find files in the subtree of /usr/include which contain a string “printf” in them. (Show solution) #!/bin/sh find /usr/include -exec grep -q printf {} \; -print # or find /usr/include | xargs grep -l printf # or find /usr/include -exec grep -l printf {} + 9. Using sed select only the column with the file size from the output of ls -l. (Show solution in sed) #!/bin/sh ls -l | sed '1d; s/^$$[^ ]* *$$\{5\} .*/\1/' (Show solution in ed) #!/bin/sh # Backslash in front of END prevents shell from performing any # expansions in the text before END. ls -l > /tmp/lsl.$$ ed /tmp/lsl.$$<<\END 2,g/^/s/^$$[^ ]* *$$\{5\} .*/\1/ %p Q END rm /tmp/lsl.$$
10. Using sed transform the lines of /etc/passwd to the form of uid (login).
(Show solution in sed)
#!/bin/sh sed 's/^$$[^:]*$$:[^:]*:$$[^:]*$$.*$/\2 (\1)/' /etc/passwd (Show solution in ed) #!/bin/sh # Backslash in front of END prevents shell from performing any # expansions in the text before END. # If wish to store the results into the input file, # we can add commmands w and q. cp /etc/passwd /tmp/passwd.$$ed /tmp/passwd.$$<<\END g/^/s/^$$[^:]*$$:[^:]*:$$[^:]*$$.*$/\2 (\1)/ %p Q END rm /tmp/passwd.$$ ### Homework: 1. Using grep select those lines from countrycodes_en.csv where the first two characters in the three character code (Alpha-3 code, 3rd column) are not the same as the two characters in the two character code (Alpha-2 code, 2nd column). [1 point] 2. In an HTML file (take this web page for example) titles are enclosed by a pair of <hi>, </hi> tags, where i is a digit 1 to 6. Find all lines with this pair of tags (the digit has to be same in both the opening and closing tag). [1 point] 3. Using sed replace in the input text file all occurrences of characters '&', '<', '>' with their HTML entities (i.e. replace '&' with '&amp;', '<' with '&lt;', and '>' with '&gt;'). Call sed just once. [1 point] ## 7th practical (4th April, 2018) Commands: sed, nl. 1. Using sed put character # to the beginning of lines 10 to 20 in file /etc/passwd. (Show solution in sed) #!/bin/sh sed '10,20s/^/#/' /etc/passwd (Show solution in ed) #!/bin/sh ed /etc/passwd <<\END 10,20s/^/#/ %p Q END 2. Using sed put character “o” to the beginning of each odd line in file /etc/passwd and “e” to the beginning of each even line in this file. (Show solution in sed) #!/usr/bin/sed -f # This is a sed script, use with sed -f # Usage: sed -f <script> /etc/passwd s/^/o/ n s/^/e/ (Show solution in ed) #!/bin/sh # The modified file is not written (it would not be a good idea in # case of /etc/passwd). ed /etc/passwd <<\END g/./s/^/o/\ +1s/^/e/ %p Q END 3. Using sed insert an empty line between each two lines of /etc/passwd. (Show solution in sed) #!/bin/sh # The b command ensures that nothing will be added after the last # line. # Command a performs the actual empty line append. sed 'b; a\ ' /etc/passwd # Equivalently we can do it in the following way with i (the addresses ensure that nothing is added before the first line): sed '2,i\ ' /etc/passwd (Show solution in ed) #!/bin/sh # Backslash in front of END prevents shell from performing any # expansions in the text before END. # # The backslash after i command is there because it is a command # within global g command, otherwise there would be no backslash after # i. ed /etc/passwd <<\END 2,g/.*/i\ \ . %p Q END 4. Using sed reverse the order of lines in the input. (Show solution in sed) #!/bin/sh # We shall try it with nl /etc/passwd to see the difference. nl /etc/passwd | sed -n 'G; p; h' (Show solution in ed) #!/bin/sh # Backslash in front of END prevents shell from performing any # expansions in the text before END. # The backslash after i command is needed because it is part of g # command. ed /etc/passwd <<\END g/^/m0 %p Q END 5. Using sed print only even lines from file /etc/passwd. (Show solution in sed) sed -n 'n; p' /etc/passwd (Show solution in ed) #!/bin/sh # Observe the backslash before string END which prevents shell # expansion in the text. # Command g performs the following commands on each line. # 1) +1p prints the next line and # 2) .s/^// unmarks (the next) line so that g skips it. # the last line is omitted (we assume that the file contains at least # two lines), because for the last line +1 would be behind the end of # file. # # If we would use s///, then instead of an empty regexp g would fill # in its own regexp and even lines would be deleted. If the changes # are not about to be saved, this is not an issue. nl /etc/passwd > /tmp/passwd.$$ ed /tmp/passwd<<\END 1,$-1g/^/+1p\ .s/^// Q END rm /tmp/passwd.$$ 6. Using sed insert #!/bin/sh as the first line of a given file. (Show solution in sed) #!/usr/bin/sed -f # This is a sed script, use with sed -f 1i\ #!/bin/sh (Show solution in ed) #!/bin/sh ed inputfile<<\END 1i #!/bin/sh . w q END 7. Write a sed script which removes comments from a shell script, in particular: 1. If the first line has a special form of ^#!.*, then it should not be removed. 2. Lines which start with # (or have only white characters before the first #) should be deleted from the output. 3. Lines which contain some other characters before # should be modified to keep only the part before the # character. 4. For simplicity ignore the fact that in a shell script the # character can occur escaped, in quotation marks, apostrophes or in other contexts. Assume that each # in the input indeed starts a comment. (Show solution in sed) #!/usr/bin/sed -f # This is a sed script, use with sed -f # Skip the first line if it has a special form of #! 1{ /^#!/n } # Delete lines starting with # /^[[:space:]]*#/d # Deletes comments from the remaining lines (ignores the fact that # # can be a part of a string in quotation marks, apostrophes etC.). s/#.*// (Show solution in ed) #!/bin/sh # We first deal with all lines except the first one. # - The first command deletes the lines having nothing before the # comment # character. # - The second command deletes everything from # to the end of line # from each of remaining lines. # Then we resolve the first line. # - We first delete the first line if it contains a comment having # nothing before # and does not have ! after #. # - We delete the first line if it contains only # and nothing else. # - Finally we remove everything from # to the end if there is not #! # at the beginning of line. ed inputfile.sh <<\END 2,g/^[[:space:]]*#/d 2,g/#/s/#.*// 1g/^[[:space:]]*#[^!]/d 1g/^[[:space:]]*#/d 1v/^#!/s/#.*// w q END ### Extra assignments (without solutions) 1. Write a sed script which replaces all C style comments (i.e. /* */) to C++ style comments (i.e. //). Be aware of the following facts: • C style comments can be on a single line, but they can be multi-line as well. • If there is something after a C style comment, you have to move it to the next line. • There can be multiple C style comments on a single line. You may assume that the input file does not contain other occurrences of /* and */ except those which delimit a comment, that the comments are not nested and that the comment delimiters are well paired. For example aaa /* bbb */ ccc /* ddd */ eee /* fff ggg */ hhh /* iii */ jjj kkk /* lll mmm nnn */ ooo  would be turned into aaa // bbb ccc // ddd eee // fff // ggg hhh // iii jjj kkk // lll // mmm // nnn ooo  The output of your script may include additional spaces (or newlines), but the output should be valid. 2. We define a number in a text file as a string which consists of digits 0-9 with possible leading sign + or - which is delimited with a blank character (a space, tab, begining of line, end of line, etc.). I.e. line 12 -31 aaaa b01 001 a-13  contains three numbers 12, -31 and 1 (leading zeros are ignored). Write shell commands which count the number of pairwise different numbers in a given file in absolute value (i.e. leading zeros are ignored, signs are ignored, in particular -31, 31, 0031, +0000031 all count as the same number in absolute value 31). Motivation: A CNF file such as formula.cnf consists of a header line and a list of clauses where numbers denote number of variables in a conjuctive normal form (CNF) formula. The commands from exercise should count the number of different variables in the formula. Exercise is a little bit more general. 3. Write a sed script which reads a decimal number n from the input and writes sequence 0, 1, ..., n to the output. Each number is on a separate line. 4. Write a sed script which outputs the last ten lines of the input. ### Homework 1. Assume that the input lines contain only characters '(' and ')', write a sed script which checks whether each input line contains string of well paired brackets. The correct lines are replaced with 'ok' string, the wrong lines with 'wrong' string. [1 points] 2. Consider a text file in which paragraphs are separated with an empty line. Write a script for sed which joins the lines of each paragraph to a single line (with space as a separator). [1 points] 3. Write a sed script which selects the middle character from each line. If the number of characters on a line is odd, it is replaced with the middle character. If the number of characters on a line is even, the line is replaced with one of the two middle characters. [1 point] ## 8th practical (11th April, 2018) Command: ed. 1. Using ed output the last ten lines of /etc/passwd. (Show solution) #!/bin/sh ed /etc/passwd <<\END -9,p Q END 2. Solve the following exercises also in ed: 9 and 10 in 6th practical, 1 to 7 in 7th practical. For solutions see each of these exercises. 3. Run vimtutor ### Extra assignments (without solutions) 1. Assume the input file contains words separated with spaces where each word consists only of characters in English alphabet ([:alpha:]) Write a sed script which reverses the order of words on each line. ### Homework 1. Write a script for ed which moves all lines starting with character '#' to the end of the input file. [1 point] 2. File morse.csv contains a translation of English alphabet into the Morse code. Write a script for sed or ed which translates this file into another file translate.sed which is a sed script translating an English text into the Morse code. In other words, the input to your script is the file morse.csv. The output of your script is the file translate.sed. We can use translate.sed as a script for sed to translate an English text into the Morse code as follows: sed -f translate.sed input >output You can expect that the text to be translated is written only using letters of English alphabet (lower or upper case) and words are separated with spaces. When translated into the Morse code, the characters are separated with '/' (a slash) and words are separated with '//' (two slashes). [2 points] ## 9th practical (18th April, 2018) Commands: read, control structures (for, while, if, case, etc.), printf, expr. Other: Variables and expansion, redirection, positional and special parameters/variables (#, 0, n, {n}, shift, "@", set -, ?,$$,$!), command combination using &&, command grouping and subshell invokation, back apostrophes and $() expansion. 1. Write a shell script which expects two parameters, a file name and a number. The script deletes the line with given number from given file. (Show solution) #!/bin/sh ed "$1" <<END  ${2}d w q END 2. Write a script which expects n+1 parameters, the first parameter contains a number x between 1 and n, the script then outputs the (x+1)st parameter. For instance script 2 a b c outputs b, i.e. the third parameter (or the second if we count starting from a. (Use shift with a parameter.) (Show solution) #!/bin/sh shift "$1" echo "$1" 3. Write script reverse which writes its parameters to standard output in reversed order. For example reverse a b c d outputs d c b a. (Show solution) #!/bin/sh for x do y="$x $y" done echo "$y"
4. Write a script max which selects the maximum of its numeric parameters. For example max 1 15 7 19 11 prints 19.
(Show solution)
#!/bin/sh # Without test, using sort for x do    echo "$x" done | sort -nr | head -n 1 # Using sort, now we assume that the parameters are numbers without spaces: echo "$@" | tr " " '\n' | sort -nr | head -n 1 # Using test m="$1" shift for x do if [ "$x" -gt "$m" ] then m="$x"    fi done echo "$m" # Using shell arithmetic expansion m="$1" shift for x do    m=$((x>m ? x : m)) done echo "$m"
5. Write a script which expects one or two number parameters. The script then pads the first number to number of digits given in the second parameter. If the second parameter is not specified, it is considered to be 5. (Use printf and expansion ${variable:-word}.) (Show solution) #!/bin/sh printf "%${2:-5}d\n" $1 6. What the following lines write to the screen? a="hello"; { echo$a ; a="hi" ; echo $a ; } ; echo$a b="hello"; ( echo $b ; b="hi" ; echo$b ; ) ; echo $b x="hello"; x="hi" sh -c 'echo$x'; echo $x y="hello"; y="hi"; sh -c 'echo$y'; echo $y z="hello"; sh -c 'echo$z' export u="hello"; sh -c 'echo $u' 7. What is the following command doing? a=1;b=2; { a=LEFT; echo$a >&2; } |\ { b=RIGHT; echo $b; tr 'A-Z' 'a-z';}; echo "A=$a,B=$b" And how is it different from: a=1;b=2; { a=LEFT; echo$a; } |\ { b=RIGHT; echo $b; tr 'A-Z' 'a-z';}; echo "A=$a,B=$b" ### Extra assignments (without solutions) Extra assignments can use any commands 1. Assume the input file is in form of a csv file where columns are separated with a comma. Write a shell script which accepts two parameters, a file and a number of a column. The script then outputs a median of the numeric values in the given column within the given file. You can also enhance the script to compute average and standard deviation of the numeric values in the column. 2. We define Fibonacci words as follows: Word $$S_0=0$$, word $$S_1=01$$, then word $$S_n=S_{n-1}S_{n-2}$$ (i.e. concatenation of words $$S_{n-1}$$ and $$S_{n-2}$$). Write a shell script which given a number $$n$$ writes $$n$$-th Fibonacci word. Allow two optional parameters which specify characters to be used instead of 0 and 1. 3. Write a shell script which reads a file with a number on each line and sorts it. Each number from the input appears only once in the output, but the script computes the number of occurrences of each number within the input file and appends this frequency to each number as another column of the output file. ### Homework 1. Write a script which works as cp but with reversed order of parameters (the target is the first parameter, note that cp accepts more than two parameters). [1 point] 2. Write a script which expects three parameters, a file name, a number n and a character c. The script then performs a circular shift of each input line (input is in the file specified in the first parameter). The lines are considered to be split into fields separated with character c and the number of elements to shift is given in number n. For example if the script is called like script /etc/passwd 2 : then each line of /etc/passwd is transformed from the initial format login:*:uid:gid:full name:home dir:shell to the following form: uid:gid:full name:home dir:shell:login:* The modified contents of the input file is written to the standard output, the file itself is left untouched. [2 points] ## 10th practical (25th April, 2018) Commands: read, expr, dirname, basename. 1. Write a shell script which expects file paths in its parameters. The paths in parameters can be relative. For each path in parameter the script outputs the full path of the file. (Show solution) #!/bin/sh for x do directory=$(dirname "$x") directory=$(cd $directory && pwd) echo$directory/$(basename$x) done
2. Will the following script correctly compute the number of lines in /etc/passwd?
count=0; cat /etc/passwd | while read x do    count=expr $count + 1 done echo$count
And what about the following script?
cat /etc/passwd | {    count=0    while read x    do       count=expr $count + 1 done echo$count }
And what about the following script?
count=0 while read x do    count=expr $count + 1 done </etc/passwd echo$count
And finally, what about the following script?
count=0 while read x </etc/passwd do    count=expr $count + 1 done echo$count
What is the difference between all these scripts?
3. Write a shell script which renames all files in the current directory to files with the same name but where the upper case characters are changed into lower case. (E.g. file BACKUP.ZIP would be renamed to backup.zip.)
(Show solution)
#!/bin/sh for x in *[A-Z]* do    mv "$x" "echo$x | tr '[:upper:]' '[:lower:]'" done
4. Write a shell script which renames all files in the current directory with suffix “.jpeg” to the files with the same name but with their suffix changed to “.jpg”.
(Show solution)
#!/bin/sh # A variant using special variable expansion (not for too long in # the standard though). for x in *.jpeg do    mv "$x" "${x%jpeg}jpg" done # A variant using sed: for x in *.jpeg do    mv "$x" "$(echo $x | sed 's/\.jpeg$/.jpg/')" done # A variant using basename and dirname. for x in *.jpeg do    mv "$x" "$(dirname "$x")/$(basename "$x" .jpeg).jpg" done 5. Write a shell script which receives three parameters, all of them are numbers. The script outputs a list of numbers separated with spaces. The numbers start with$1, end with $2 and they are increased with step given in$3. If the third parameter is missing, then step 1 is considered by default.
(Show solution)
#!/bin/sh if [ $# -lt 2 ] || [$# -gt 3 ] then    echo Bad number of parameters.    exit 1 fi step=${3:-1} x=$1 while [ $x -le$2 ] do    printf "$x " x=expr$x + $step done 6. The same as the previous exercise, but now each number is on a separate line and they are justified to the number of digits of the longest number (i.e.$2). The numbers are justified with leading zeros, i.e.
script 0 10 2
would output:
00
02
04
06
08
10

(Show solution)

### Homework

1. Write script sum which sums the numeric values of parameters and writes the result to standard output. For example sum 4 13 112 7 outputs 136. [1 point]
2. Write a script rmexcept, which expects n+1 parameters where the first parameter is a directory and the rest of parameters contain shell masks specifying file names. The script removes all files whose names do not match any of the given shell masks from the given directory. The files whose names match one of the masks, are left untouched (in particular they cannot be moved anywhere else and then back). E.g. calling rmexcept . '*.jpg' '*.png'
would remove all files whose suffix is neither jpg nor png from the current working directory. [2 points]

## 11th practical (1st May, 2018)

1. Write a shell script which expect the lines of the input in the following form:
goal : z1 z2 .... zn
where goal, z1, z2, ..., zn are file names (paths). The script will list the goals which are older than any file of z1, z2, z3 ..., zn (subgoals) on the same line. (Note that parameter -nt of test is not standard, you can use find with parameter -newer or ls with parameter -t.)
(Show solution)
#!/bin/sh    # The solution would not work if there would be spaces in file    # names, but then we would have to specify how to distinguish a    # space separating two file names and a space being a part of a    # filename, so we will ignore spaces in filenames for now. while read goal colon files do    set -- $files if [ "$(find "$@" -newer "$goal" -prune)" ]    then       echo $goal fi done # An alternative solution using ls -t: while read goal colon files do if [ "$(ls -t "$goal"$files | head -n 1)" != "$goal" ] then echo$goal    fi done
2. Write a script newfrom which receives pathes to two directories DIR1 and DIR2. The files in DIR1 which either are not in DIR2, or they are newer in DIR1 than in DIR2 are copied from DIR1 to DIR2. The script works recursively also on the subdirectories of DIR1 and DIR2. The script accepts an optional switch -n, if this switch is provided, the script does not actually copy anything, just lists the files which would be copied had -n switch not been provided. In case DIR2 does not exist, it is created. The same is true for subdirectories.
(Show solution)
#!/bin/sh usage() {    echo "Usage: $0 [-n] dir1 dir2" exit 1 } if [ "("$# -eq 3 ")" -a "(" "$1" == "-n" ")" ] then DRY_RUN=1 shift else DRY_RUN=0 fi if [$# -ne 2 ] then    usage fi DIR1="$1" DIR2="$2" if [ ! -d "$DIR1" ] then echo "$DIR1 is not a directory"    usage fi if [ ! -d "$DIR2" ] then if [$DRY_RUN -eq 0 ]    then       cp -pR "$DIR1" "$DIR2"    else       echo cp -pR "$DIR1" "$DIR2"    fi else    ( cd "$DIR1" && find . -type d -path "*/*" ) >/tmp/tempfile1.$$while read line do rel_path="echo "line" | cut -d / -f 2- " if [ ! -d "DIR2/rel_path" ] then if [ DRY_RUN -eq 0 ] then mkdir "DIR2/rel_path" else echo "mkdir DIR2/rel_path" fi fi done </tmp/tempfile1.$$ ( cd "$DIR1" && find . -type f -path "*/*" ) >/tmp/tempfile1.$$while read line do rel_path="echo "line" | cut -d / -f 2- " if [ "find "DIR1/rel_path" -newer "DIR2/rel_path"" ] then if [ DRY_RUN -eq 0 ] then cp -p "DIR1/rel_path" "DIR2/rel_path" else echo "cp -p DIR1/rel_path DIR2/rel_path" fi fi done </tmp/tempfile1.$$ fi rm /tmp/tempfile1.
3. Write script cutreorder which behaves similarly to cut, it also has parameters -d and -f with the same format, meaning, and default values as cut. The rest of the parameters contains file names (if missing, standard input is used). The difference from cut is that it reorders the columns in the output so that they are in the order specified with parameters -f. (Recall that in cut the columns in the output are in the same order as in the input file.) For instance:
• cutreorder -d : -f 3,1 /etc/passwd would print /etc/passwd in two columns uid:login.
• cutreorder -d : -f 3,3-5,1 /etc/passwd would print columns uid:uid:gid:full name:login.

(Ukázat řešení)
#!/bin/sh # Processing parameters # The default delimiter is tab (put here literaly between the quotation marks. delim=" " fields="" for x in 1 2 do    case "$1" in -d) delim="$2"; shift 2;;       -d?) delim="${1#-d}"; shift;; -f) fields="$2"; shift 2;;       -f*) fields="${1#-f}"; shift;; esac done if [ "$fields" = "" ] then    # Nothing to output    exit 0 fi # Check if the standard input should not be used if [ $# -eq 0 ] then set -- - fi # Modify the fields into a form suitbale to be used in a for cycle fields="echo$fields | tr , " "" # Process the files for a_file do    cat $a_file | while read line do result="" # Process the fields in the input and output for f in$fields       do          if [ -n "$result" ] then result="$result$delim" fi result="$result$(echo "$line" | cut -d $delim -f$f)"       done       echo "$result" done done ### Extra assignments (without solutions) 1. Write a shell script which selects n longest lines from the files passed to it in parameters. Each output line is prepended with its number within the file (format line number: line content). The lines in the output are in the same order as they are in the input file, i.e. as if sorted by their line number. Parameter n is 5 by default but it can be specified in the first parameter as -n10 or in the first two parameters as -n 10 (this is for the case when n=10). The syntax is thus same as in case of head. If the file names are missing in parameters, standard input is used. If at least two files are specified in parameters, each list of lines is preceded by the corresponding file name. ### Homework 1. Write a script csvcut, which accepts two or more parameters with the following meaning: The 1st parameter is a delimiter character delim, the 2nd parameter is a string specifying a column in a CSV file, the 3rd and following parameter specify file names (if these names are missing, the script reads standard input). The script assumes that the input files are CSV files with delim as a delimiter character. Moreover the script assumes that the first line of each of these files contains a header. The script cuts the specified column from the input files, i.e. it cuts the column whose name is equal to the 2nd parameter of the script, regardless the possible quotation marks. For example • csvcut ';' Measure calories.csv cuts the second column from file calories.csv that is it would be equivalent to cut -d ';' -f 2 calories.csv). • csvcut ';' Food calories.csv cuts the first column from the same file although the name is surrounded with quotation marks. • [3 points] ## 12th practical (9th May, 2018) Command: awk. 1. Write an awk script which outputs each tenth line of the input. (Show solution) #!/bin/awk -f (NR % 10) == 0 2. Write an awk script which changes the characters in login field of /etc/passwd file to upper case. (Show solution) #!/bin/awk -f BEGIN { FS=":"; OFS=":" } {$1=toupper($1); print } 3. Write an awk script, which reverses the order of words at each line of the input (writes the words in opposite order). (Show solution) #!/bin/awk -f { for (i = NF; i > 1; i --) { printf ("%s" OFS,$i);    }    printf ("%s" ORS, $1); } 4. Write an awk script which numbers the lines of the input (similarly to nl). (Show solution) #!/bin/awk -f { print NR " "$0; }
5. Write an awk script which lists random numbers between 0 and 1, the number of random number to be output is given in the parameter.
(Show solution)
#!/bin/sh # This is a shell script which encapsulates the awk script. # The number of random numbers to be output is passed as the value of n. awk -v n="$1" 'BEGIN { srand(); for (i=0; i<n; ++i) { print rand (); } }' 6. Write an awk script which prints the number of seconds since beginning of the UNIX era and terminates. Hint: Use srand(). (Show solution) #!/bin/awk -f BEGIN { srand(); printf("%d\n", srand()); } 7. Write an awk script which computes the number of HTML links in an HTML file (i.e. counts the tags <a). (Show solution) #!/bin/awk -f # RS will be "<", because that is how HTML is separated. BEGIN { RS = "<"; cnt = 0; } ($1 ~ /^a$/) { ++ cnt; } END { print cnt; } 8. Write an awk script which expects an HTML file at the input and lists the web addresses which the file references (using tag <a href="address">. Note that the tag can be split over several lines and that there can be spaces around a and href. Hint: Use suitable RS and FS values. (Show solution) #!/bin/awk -f # The solution is a bit complicated so that it works even if there are # other parameters to a between a and href. Otherwise it would be # enough to take i=1. BEGIN { RS = "<"; FS = "="; }$1 ~ /^[[:blank:]]*a[[:blank:]]+/ {    i=1    while (i <= NF)    {       if ($i ~ /[[:blank:]]+href[[:blank:]]*$/)       {          split ($(i+1), a, "[[:blank:]]*\"[[:blank:]]*"); print a[2] } else if ($i ~ />/)       {          break;       }       ++i;    } }
9. Write an awk scripts which outputs the last 10 lines of the input file.
(Show solution)
#!/bin/awk -f BEGIN {    konecBufferu = 0; } {    buffer [konecBufferu] = $0; konecBufferu = (konecBufferu + 1) % 10; } END { if (NR > 0) { if (NR < 10) { indexVBufferu = 0; } else { indexVBufferu = konecBufferu; } do { print buffer[indexVBufferu]; indexVBufferu = (indexVBufferu + 1) % 10; } while (indexVBufferu != konecBufferu) } } 10. Let us define a word as a maximal sequence of alphanumeric characters (from class [:alnum:]). Write an awk script which reads a text and after it finishes the reading, it prints for each word in the text file the number of its occurrences within the input text. (Show solution) #!/bin/awk -f BEGIN { FS="[^[:alnum:]]" } { for(i=1; i<=NF; ++i) { if($i!="")       {          ++count[$i] } } } END { for(word in count) { printf("%s: %d\n", word, count[word]) } } 11. Write an awk script which expects a file in the csv format at the input. This means that each line of the input line consists of several fields separated with a comma ",". The first line is the header line, it contains the column names. The first column of the subsequent lines contains a name, the rest of columns contain numbers. The script adds a new column called “Sum” (in the header line), which on each line contains the sum of the number fields. For example in case of input Name,pts1,pts2,pts3 Hynek,3,12,9 Jarmila,7,34,1 Vilém,8,27,0  the following output is produced by the awk script: Name,pts1,pts2,pts3,Sum Hynek,3,12,9,24 Jarmila,7,34,1,42 Vilém,8,27,0,35  (Show solution) #!/usr/bin/awk -f BEGIN { FS=","; OFS="," } FNR == 1 {$(NF+1)="Sum"; print } FNR > 1 {  $(NF+1)=0 for (i=1; i < NF; i++) {$NF+=$i } print } ### Homework: When you set the record separator to the empty string, it means that the records are separated with an empty line. 1. Write an awk which reads a text and if it encounters a line starting with # (i.e. a comment line), it will add # followed by a space to the beginning of each following line until the end of the paragraph (paragraphs are separated with an empty line). The lines which already start with # are not modified. [1 point] E.g. the following input a b c d e f # g h i j k l m n o p q r s t u # v w x # y z 1 2 3 4 # 5 6 7 8 9 0  would be modified into a b c d e f # g h i # j k l # m n o p q r s t u # v w x # y z 1 # 2 3 4 # 5 6 7 # 8 9 0  2. Write an awk script which formats the lines of each paragraph in a file to a given number of characters (without splitting the words). The awk script takes a numeric parameter n (e.g. using -v n=...). The rest of parameters are files to be processed. The script considers the input files to be text files where paragraphs are separated with an empty line. Within each paragraph the lines are formated to be at most n characters long. [2 points] ## 13th practical (23rd May, 2018) Commands: trap, ps, kill, sleep, date. 1. Write a script which prints every line it reads from the standard input. If the script receives signal KILL (9), then it naturally stops. If the script receives one of signals TERM (15), QUIT (3), INT (2) (the one after CTRL-C), the script does not stop, but it only prints out the number of singal it received. In case of INT (2) the script moreover prints out the last line which was read from the input. (Show solution) #!/bin/sh trap "echo 15" TERM trap "echo 3" QUIT trap 'echo 2; echo$x' INT while read x do    echo $x done 2. Write a script which expects two parameters, a signal and a program name. The script then sends the signal to all processes of the program. You can assume that the program name does not contain any weird characters (it can contain a space, however). That is, the aim is to create something like killall, but killall itself cannot be used as it is not a standard command. I suggest to look at parameter -o of command ps. (Show solution) #!/bin/sh [$# -eq 2 ] || {    echo "Expecting two parameters, a signal and a program name.";    exit 1 } signal="$1" program="$2" ps -e -o pid= -o comm= |\ sed -n 's/^ *$$[[:digit:]]\{1,\}$$ \{1,\}'"$program"'$/\1/p' |\ xargs kill -$signal # The same can be achieved using a while cycle. ps -e -o pid= -o comm= | while read pid cmd do if [ "$cmd" == "$2" ] then kill -$signal $pid fi done 3. Write a script which expects one parameter specifying number of seconds. The script then waits for given number of seconds before it finishes. When the script receives signal SIGINT (2 - Ctrl-C), then it writes down how many seconds did it wait until now. When the script receives signal SIGQUIT (3 - Ctrl-\), then it writes down how many seconds remain. When the script receives SIGTERM (15), then it writes down how many seconds did it wait, how many seconds remain and stops. The exit code of the script should be 0 if it finishes in the required time, or the remaining number seconds if it was terminated earlier (due to SIGTERM (15)). (Show solution) #!/bin/sh die() { echo "$1"    exit 1; } now() {    awk 'BEGIN { srand(); print srand(); }' } [ $# -eq 1 ] || die "Wrong number of parameters" expr "$1" : '^[[:digit:]]*$' >/dev/null || die "Parameter is not a number" start=$(now) stop=$((start +$1)) trap 'echo $(($(now) - start))' 2 trap 'echo $((stop -$(now)))' 3 trap 'echo $(($(now) - start)); kill $! 2>/dev/null; exit' 15 while [ "$(now)" -lt "$stop" ] do sleep$((stop - $(now) )) & wait$!    [ $? -gt 128 ] && kill$! 2>/dev/null done # It would be sufficient to use sleep 1 in the above cycle, but then # the process would wake up each second. # sleep $((stop -$(now) )) is not enough by itself because of SIGTERM, # if the shell process running the script receives SIGTERM when sleep # is running, then it first lets sleep to finish and then it processes # the signal using trap. (This is not the case of Ctrl-C as there the # signal is received by sleep.) That is why sleep is being run on # background and then it is waited for the child process to finish. # After wait the signal is processed and SIGTERM is passed to the # sleep process if necessary. Wait returns 128 if it was terminated # due to a signal.
4. Sample exam question with a solution

### Homework

1. Write a script which will indent lines in the input files depending on how deep the brackets are nested. The parameters to the script are an indentation character c and a number of characters per level n, these parameters are followed by a list of files (if no files are provided, standard input is used). The script then reads line by line and on each line it removes the white characters at the beginning of line and replaces them with k*n characters c where k is the level of bracket nesting. Consider normal brackets (), curly brackets {} and square brackets []. For example the input file
a ( b
c d [ e ] f [
g h { j (
k ) } l m
n ] o ) p
q r

would be modified as follows if the script is run with parameters c='.' and n=1:
a ( b
.c d [ e ] f [
..g h { j (
....k ) } l m
..n ] o ) p
q r

You can assume that the input has well paired brackets. A space or a tab character must be allowed as character c. [3 points]

## Additional homework (if you do not have enough points).

Further exercises will be added as needed.

1. Write a script which receives number of seconds sec, a program name prg followed by its arguments. The script runs program prg with its arguments and then waits for it to finish. If the program does not finish within sec seconds, the script sends signal TERM to prg. If after at most another 5 seconds the program still does not finished, the script kills it by sending signal KILL to it. [3 points]
2. Let us define a word to be a maximal nonempty sequence of alphanumeric characters and underscores (i.e. matching BRE [[:alnum:]_]\{1,\}). Consider the word to name a variable (function, type, etc.) in a programming language. We say that the word is in camel case if the subwords are introduced with an upper case letter (such as MyVerySpecialVariable). The word can start with an upper case character (UpperCamelCase) or a lower case character (lowerCamelCase). For simplicity, we assume underscores are not present in camel case. Another common naming style is snake case which are in lower case only and subwords are separated with underscores (such as my_very_special_variable, snake_case). Your task is to write two scripts which translate one style to another (camel2snake and snake2camel). In particular the scripts work as filters, i.e. they read the standard input and write to the standard output.
• Script camel2snake translate all words which are in UpperCamelCase or lowerCamelCase are translated to snake case (words containing underscores are left untouched).
• On the other hand script snake2camel translate every word in snake case into camel case. Default is UpperCamelCase. If option -u is present, then it translates into lowerCamelCase. Words not in snake case are left untouched (those containing capital characters).
[4 points, 2 for each script].