screen basics

For that, you might use the nohup command. But what if you want to start a command-line session on one computer and then go home and resume that session?

For something like that, you’ll want to use screen. Screen is also really handy because you can have multiple shells running in one terminal window. Here’s the basics of how to use screen on a Linux/Ubuntu machine. First,

create a session:

su - root
apt update
apt install screen
screen -S sessionname

It’s good to choose a descriptive session name. For example, before I switched to using Gmail I used mutt. So I would often start a session with the command “screen -S mutt” in one terminal window. If I went home, I could attach to that session from home, so I never needed to start-up or shutdown mutt.

You can easily have 10 interactive shells (numbered from 0 to 9) open in one terminal window. When you start a new session, you’ll be in shell 0. To create a new shell To create a new interactive shell, press Control-a c Switching between shells is easy too. To switch between shells press Control-a # where # is the number of the shell.

wat sessions are currently running?

screen -ls

resume session:

screen -d -R sessionname

will resume a session that you started in a different location. Technically this command is doing some special things. If you started the session somewhere else, the “-d” option will “detach” it at the other location, and “-R” will reattach your current terminal window to the session. If sessionname hasn’t been created, it will create the session for you. So the command above will migrate a session to your current window and disable the session at other locations. That’s usually what you want. To exit a session, it’s easiest to exit all the shells in the session. If you want some more info on screen: Control-a ? will give help inside of screen.
Typing “man screen” at a Linux command-line will show more help on screen.
There are web pages with more info on screen. (creditz)

screen; # start screen
/scripts/stress_loop.sh; # start the script

# detach (Ctrg+A then D) and logout (Ctrl+D) and re-login

screen -ls; # list all running screens
There is a screen on:
3028.pts-0.debian (05/16/2017 01:24:19 PM) (Detached)
1 Socket in /var/run/screen/S-root.

screen -R 3028.pts-0.debian; # resume this screen
Ctr+C - cancel foreground job
exit; # exit screen

# cool eh?
# the user can name the screens

screen -S loop; # start screen with name 3067.loop

When screen is called, it creates a single window with a shell in it (or the specified command) and then gets out of user’s way so that the user can use the program as the user normally would.

Then, at any time, the user can create new (full-screen) windows with other programs in them (including more shells), kill existing windows, view a list of windows, turn output logging on and off, copy-and-paste text between windows, view the scrollback history, switch between windows in whatever manner the user wish, etc.

All windows run their programs completely indepen‐ dent of each other.

Programs continue to run when their window is currently not visible and even when the whole screen session is detached from the user’s terminal.

When a program terminates, screen (per default) kills the window that contained it.

If this window was in the foreground, the display switches to the previ‐ ous window; if none are left, screen exits. Shells usually distinguish between running as login-shell or sub-shell.

Screen runs them as sub-shells, unless told otherwise (See “shell” .screenrc command).

Screen version 4.02.01 (GNU) 28-Apr-14

Copyright (c) 2010 Juergen Weigert, Sadrul Habib Chowdhury
Copyright (c) 2008, 2009 Juergen Weigert, Michael Schroeder, Micah Cowan, Sadrul Habib Chowdhury
Copyright (c) 1993-2002, 2003, 2005, 2006, 2007 Juergen Weigert, Michael Schroeder
Copyright (c) 1987 Oliver Laumann

This program is free software; 😀

Send bugreports, fixes, enhancements, t-shirts, money, beer & pizza to screen-devel@gnu.org
Capabilities: +copy +remote-detach +power-detach +multi-attach +multi-user +font +color-256 +utf8 +rxvt +builtin-telnet
[Press Space or Return to end.]

Manpage: screen.man.txt

search & terminate a program

that the user needs to terminate a program… like the terminator… (but in this version not destroying a program, but simply politely telling it’s instance to quit)

it could happen if a ssh session was started as a job, and then the terminal was quit… it keeps running in the background and “jobs” does not show it

# this outputs the first line of ps (what is what)
# and the line that contains all processes that have to do with ssh
# the process in question is marked in orange
ps uax | egrep 'ssh|PID'
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root       815  0.0  0.0  15852  6692 ?        Ss   14:18   0:00 /usr/sbin/sshd -D
username   941  0.0  0.0   5852   472 ?        Ss   14:18   0:00 /usr/bin/ssh-agent x-session-manager
username  2942  0.0  0.0   5852  3780 ?        S    14:58   0:00 /usr/bin/ssh-agent -D -a /run/user/1000/keyring/.ssh
username 14595  0.0  0.0  15788  5812 ?        S    20:16   0:00 ssh username@domain.com
username 14698  0.0  0.0   6208   824 pts/1    S+   20:18   0:00 grep -E ssh|PID

# now terminate the process
# SIGINT "interrupt" like Ctrl+C "cancel"
kill -SIGINT 14595

show user’s limits

the user could limit the amount of files a user may open at the same time.

the user could limit the amount of RAM a user is allowed to use.

users may reduce their limit themselves but not set them to unlimited (only root can do that).

see also: https://dwaves.de/2017/06/09/lpic-1-102-110-1-perform-security-administration-tasks/

user@debian:~$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 7924
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 65536
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 7924
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

basic job control and niceness (priority of processes)

while physically at the machine…

vim /scripts/stress_loop.sh;

#!/bin/bash
# test-script that does useless work but consumes a lot of CPU ressources
while true; do
echo -n "+.";
done;

chmod u+x /scripts/stress_loop.sh; # mark it runnable

/scripts/stress_loop.sh; # run it
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+...
# the user could now press Ctrg+Z - to pause it and put process into background
[1]+  Stopped                 /scripts/stress_loop.sh

fg; # resume it in foreground <- do this
bg; # resume it in background <- if the user does that
#  user will have to blindly type fg 1;

# to bring job 1 into foreground
# hit Ctrl+C - to cancel/abort foreground process
# start script as background-job without output
/scripts/stress_loop.sh > /dev/null &

jobs; # list all running jobs
[1]+  Running                 /scripts/stress_loop.sh > /dev/null &

fg 1; # bring it to foreground
/scripts/stress_loop.sh > /dev/null
Ctrl+C; # abort cancel process

# now start multiple instances from different consoles

# Ctrl+Alt+F1 - go to terminal one and login
nice /scripts/stress_loop.sh; # run with default priority

# Ctrl+Alt+F2 - go to terminal two and login
nice -10 /scripts/stress_loop.sh; # run with niceness of 10

# Ctrl+Alt+F3 - go to terminal three and login
nice --20 /scripts/stress_loop.sh; # with niceness of -20

# Ctrl+Alt+F4 - go to terminal three and login
# check out what is going on
htop


the user will see that the process with the smaller nice-level – gets more CPU resources.

go back to terminal1: Ctrl+Alt+F1 … and the user will see the script still at work… but way slower as if it was running alone.

go to terminal2: Ctrl+Alt+F2 … and the user will see – this process with niceness of +10 is running even slower.

while process on terminal3: Ctrl+Alt+F3 with niceness of -20 is running at full speed.

the user can re-adjust priority – also after process was started:

root@debian:~# renice 0 2546
2546 (process ID) old priority -20, new priority 0

the user can terminate all 3 scripts with:

killall stress_loop.sh

default nice-level is zero (0) – nice-values below zero are only allowed to be started by root.

non-root users are allowed to re-nice their processes but only to larger (lower priority) levels.

keep processes running after logout

two possibilities: nohup or screen

nohup – will write all output to a file called nohup – the user will have to append & in order to make it run in the background – before logging out.

screen – is starting a new bash – even interactive processes that are running in the foreground continue running when the user detach (Ctrg+A then D) and logout (Ctrl+D).

nohup /scripts/stress_loop.sh & # start script as background process
nohup: ignoring input and appending output to ‘nohup.out’

jobs
[1]+  Running                 nohup /scripts/stress_loop.sh &

# if the user logs out now and re-login the user will realize
jobs
# no jobs are displayed
# but
ps uax|grep loop; # it is still running
root      2986 97.1  0.2   5080  2752 ?        R    13:21   1:00 /bin/bash /scripts/stress_loop.sh

tail -f nohup; # all output gets written to this file

killall stress_loop.sh

Example: Process/Job control

sleep 600 &; # start a job and send to background
[1] 4206 # process1 is running with PID 4206 asigned

user@suse:~> echo $!; # return the PID of the last run job
4206

user@suse:~> grep ^State /proc/4206/status; # grep the status of that process
State:  S (sleeping)

user@suse:~> fg; # bring last started job to foreground
sleep 600
^Z; # send to background and stop
[1]+  Stopped                 sleep 600

user@suse:~> grep ^State /proc/4206/status; # grep the status of that process
State:  T (stopped)

user@suse:~> kill -CONT 4206; # resume the job
user@suse:~> jobs
[1]+  Running                 sleep 600 &

kill -STOP 4206; # pause process

user@suse:~> jobs
[1]+  Stopped                 sleep 600

jobs
[1]+  Getötet(killed)                sleep 600

kill -l; # list of all available signals
 1) SIGHUP       2) SIGINT       3) SIGQUIT      4) SIGILL       5) SIGTRAP
 6) SIGABRT      7) SIGBUS       8) SIGFPE       9) SIGKILL     10) SIGUSR1
11) SIGSEGV     12) SIGUSR2     13) SIGPIPE     14) SIGALRM     15) SIGTERM
16) SIGSTKFLT   17) SIGCHLD     18) SIGCONT     19) SIGSTOP     20) SIGTSTP
21) SIGTTIN     22) SIGTTOU     23) SIGURG      24) SIGXCPU     25) SIGXFSZ
26) SIGVTALRM   27) SIGPROF     28) SIGWINCH    29) SIGIO       30) SIGPWR
31) SIGSYS      34) SIGRTMIN    35) SIGRTMIN+1  36) SIGRTMIN+2  37) SIGRTMIN+3
38) SIGRTMIN+4  39) SIGRTMIN+5  40) SIGRTMIN+6  41) SIGRTMIN+7  42) SIGRTMIN+8
43) SIGRTMIN+9  44) SIGRTMIN+10 45) SIGRTMIN+11 46) SIGRTMIN+12 47) SIGRTMIN+13
48) SIGRTMIN+14 49) SIGRTMIN+15 50) SIGRTMAX-14 51) SIGRTMAX-13 52) SIGRTMAX-12
53) SIGRTMAX-11 54) SIGRTMAX-10 55) SIGRTMAX-9  56) SIGRTMAX-8  57) SIGRTMAX-7
58) SIGRTMAX-6  59) SIGRTMAX-5  60) SIGRTMAX-4  61) SIGRTMAX-3  62) SIGRTMAX-2
63) SIGRTMAX-1  64) SIGRTMAX

kill -SIGHUP 4206; # "hangup" force shell to quit all child processes

# there are three different possibilities to send the same signal to the process
kill -SIGINT 4206; # "interrupt" like Ctrl+C "cancel"
kill -INT    4206; # "interrupt" short version
kill -2      4206; # "interrupt" invoked by numeric id

kill -TERM 4206; # default for "kill" and "killall", ask the process to quit gracefully (e.g. database shutdown)
kill -15   4206; #  invoked by numeric id

kill -KILL 4206; # "kill" if process is not reacting to quit gracefully via SIGINT, the user can force quit with SIGKILL
kill -9    4206; # "kill" invoked by numeric id

what processes are currently running?

listing all running processes programs finding infos about an process.

ps aux |less; # what processes are currently running?
#            processID       virtual memory reserved
#           /                /      /"resident set size" - actual memory taken/reserved by process
#           |                |      |
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.5 185256  5964 ?        Ss   09:22   0:02 /usr/lib/systemd/systemd --switched-root --system --deserialize 23
root         2  0.0  0.0      0     0 ?        S    09:22   0:00 [kthreadd]
root         3  0.0  0.0      0     0 ?        S    09:22   0:00 [ksoftirqd/0]
root         7  0.0  0.0      0     0 ?        S    09:22   0:00 [rcu_sched]
root         8  0.0  0.0      0     0 ?        S    09:22   0:00 [rcu_bh]
root         9  0.0  0.0      0     0 ?        S    09:22   0:00 [migration/0]
root        10  0.0  0.0      0     0 ?        S    09:22   0:00 [watchdog/0]
root        11  0.0  0.0      0     0 ?        S    09:22   0:00 [watchdog/1]
root        12  0.0  0.0      0     0 ?        S    09:22   0:00 [migration/1]
root        13  0.0  0.0      0     0 ?        S    09:22   0:00 [ksoftirqd/1]
root        15  0.0  0.0      0     0 ?        S<   09:22   0:00 [kworker/1:0H]
root        17  0.0  0.0      0     0 ?        S    09:22   0:00 [kdevtmpfs]
root        18  0.0  0.0      0     0 ?        S<   09:22   0:00 [netns]
root        19  0.0  0.0      0     0 ?        S<   09:22   0:00 [perf]
root        20  0.0  0.0      0     0 ?        S    09:22   0:00 [khungtaskd]
root        21  0.0  0.0      0     0 ?        S<   09:22   0:00 [writeback]
...                                            |              \_cpu time used
                                                \_status of process

top; # like windows task manager but on console, hit Shift+M to sort for MEMORY usage
# RES = currently used RAM by process
# example: gdm (gnome3) uses 156812KByte of 156MByte RAM
top - 12:10:47 up  2:48,  2 users,  load average: 0,10, 0,03, 0,01
Tasks: 149 total,   1 running, 148 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0,0 us,  0,0 sy,  0,0 ni, 99,8 id,  0,2 wa,  0,0 hi,  0,0 si,  0,0 st
KiB Mem:   1007380 total,   791520 used,   215860 free,     2076 buffers
KiB Swap:  2103292 total,        0 used,  2103292 free.   531884 cached Mem
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 1643 gdm       20   0 1532396 156812  77164 S 0,000 15,57   0:05.59 gnome-shell
 1241 root      20   0  303928  37072  21236 S 0,000 3,680   0:00.29 Xorg
 1629 gdm       20   0  887972  32888  24700 S 0,000 3,265   0:00.44 gnome-settings-
 1286 polkitd   20   0  528168  27784   9468 S 0,000 2,758   0:00.38 polkitd
 1600 gdm       20   0  558216  13080  11356 S 0,000 1,298   0:00.06 gnome-session-b
 1243 root      20   0  280604   8780   5940 S 0,000 0,872   0:00.29 accounts-daemon
 1637 root      20   0  287564   8696   7692 S 0,000 0,863   0:00.05 upowerd
 1590 root      20   0  242652   8540   7372 S 0,000 0,848   0:00.02 gdm-session-wor
  543 root      20   0   33308   8524   8176 S 0,000 0,846   0:01.55 systemd-journal
 2164 root      20   0  102732   7828   6816 S 0,000 0,777   0:00.04 sshd
...

htop; # might not be preinstalled in user's system but is even more intuitive and colorful than top
# it has a "ascii-grafical" display of RAM and CPU usage, hit F6 and select M_RESIDENT to sort for currently reserved RAM 
  2  [                                                                         0.0%]     Load average: 0.09 0.03 0.01
  Mem[|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||     285/983MB]     Uptime: 02:50:22
  Swp[                                                                     0/2053MB]
Sort by          PID USER      PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  Command
PID             1664 gdm        20   0 1496M  153M 77164 S  0.0 15.6  0:00.00 gnome-shell --mode=gdm
USER            1665 gdm        20   0 1496M  153M 77164 S  0.0 15.6  0:00.32 gnome-shell --mode=gdm
PRIORITY        1666 gdm        20   0 1496M  153M 77164 S  0.0 15.6  0:00.28 gnome-shell --mode=gdm
NICE            1667 gdm        20   0 1496M  153M 77164 S  0.0 15.6  0:00.07 gnome-shell --mode=gdm
M_SIZE          1669 gdm        20   0 1496M  153M 77164 S  0.0 15.6  0:00.00 gnome-shell --mode=gdm
M_RESIDENT      1670 gdm        20   0 1496M  153M 77164 S  0.0 15.6  0:00.00 gnome-shell --mode=gdm
M_SHARE         1671 gdm        20   0 1496M  153M 77164 S  0.0 15.6  0:00.00 gnome-shell --mode=gdm
STATE           1672 gdm        20   0 1496M  153M 77164 S  0.0 15.6  0:00.00 gnome-shell --mode=gdm
PERCENT_CPU     1643 gdm        20   0 1496M  153M 77164 S  0.0 15.6  0:05.62 gnome-shell --mode=gdm
PERCENT_MEM     1576 root       20   0  296M 37072 21236 S  0.0  3.7  0:00.00 /usr/bin/Xorg :0 -background none -verbose -auth /run/gdm/auth-for-gdm-eJrfFw/database -seat seat
TIME            1577 root       20   0  296M 37072 21236 S  0.0  3.7  0:00.00 /usr/bin/Xorg :0 -background none -verbose -auth /run/gdm/auth-for-gdm-eJrfFw/database -seat seat
Command         1241 root       20   0  296M 37072 21236 S  0.0  3.7  0:00.28 /usr/bin/Xorg :0 -background none -verbose -auth /run/gdm/auth-for-gdm-eJrfFw/database -seat seat
                1633 gdm        20   0  867M 32888 24700 S  0.0  3.3  0:00.17 /usr/lib/gnome-settings-daemon-3.0/gnome-settings-daemon
                1634 gdm        20   0  867M 32888 24700 S  0.0  3.3  0:00.00 /usr/lib/gnome-settings-daemon-3.0/gnome-settings-daemon
                1635 gdm        20   0  867M 32888 24700 S  0.0  3.3  0:00.03 /usr/lib/gnome-settings-daemon-3.0/gnome-settings-daemon
                1636 gdm        20   0  867M 32888 24700 S  0.0  3.3  0:00.00 /usr/lib/gnome-settings-daemon-3.0/gnome-settings-daemon
                1629 gdm        20   0  867M 32888 24700 S  0.0  3.3  0:00.45 /usr/lib/gnome-settings-daemon-3.0/gnome-settings-daemon
                1291 polkitd    20   0  515M 27784  9468 S  0.0  2.8  0:00.00 /usr/lib/polkit-1/polkitd --no-debug
                1292 polkitd    20   0  515M 27784  9468 S  0.0  2.8  0:00.07 /usr/lib/polkit-1/polkitd --no-debug
                1293 polkitd    20   0  515M 27784  9468 S  0.0  2.8  0:00.00 /usr/lib/polkit-1/polkitd --no-debug
...

So how much RAM is a process actually taken?

-> check out the RSS field, ignore the VSZ

The SIZE and RSS fields don’t count some parts of a process including the page tables, kernel stack, struct thread_info, and struct task_struct.

This is usually at least 20 KiB of memory that is always resident.

SIZE is the virtual size of the process (code+data+stack).

RSS is the Resident Set Size and is used to show how much memory is allocated to that process and is in RAM.

– It does not include memory that is swapped out.
+ It does include memory from shared libraries as long as the pages from those libraries are actually in memory.
+ It does include all stack and heap memory.

VSZ is the Virtual Memory Size.
It includes all memory that the process can access, including memory that is swapped out and memory that is from shared libraries.

So if process A has a 500K binary
+ and is linked to 2500K of shared libraries
+ has 200K of stack/heap allocations of which 100K is actually in memory (rest is swapped), and has only actually loaded 1000K of the shared libraries and 400K of its own binary then:

VSZ: 500K + 2500K + 200K = 3200K (this is what the process would consume – if it fully loads itself to RAM including all of it’s shared libraries)

RSS: 400K + 1000K + 100K = 1500K (this is how much RAM is actually taken, because a process might not need to load 100% of itself and the shared libraries to RAM)

Since part of the memory is shared, many processes may use it, so if the user add up all of the RSS values the user can easily end up with more space than user’s system has.

PROCESS STATE CODES

Here are the different values that the s, stat and state output specifiers (header “STAT” or “S”) will display to describe the state of a
D uninterruptible sleep (usually IO)
R running or runnable (on run queue)
S interruptible sleep (waiting for an event to complete)
T stopped, either by a job control signal or because it is being traced
W paging (not valid since the 2.6.xx kernel)
X dead (should never be seen)
Z defunct (“zombie”) process, terminated but not reaped by its parent

Zombie process:

usually a child process returns an result like 0 = everything ok, 1 = some error

a parent process hangs or quits before child-process… and one child process terminates with some result, it can not return the results to it’s parent and turn into zombie 😀 (terminated but result not fetched by parent… so it also hangs?)

what shared libraries is the process with PID X using?

cat /proc/2176/maps; # what shared libraries is the process with PID 2176 (bash) using?
00400000-004a6000 r-xp 00000000 00:25 30785                              /bin/bash
006a5000-006a6000 r--p 000a5000 00:25 30785                              /bin/bash
006a6000-006aa000 rw-p 000a6000 00:25 30785                              /bin/bash
006aa000-006b5000 rw-p 00000000 00:00 0
01394000-015c6000 rw-p 00000000 00:00 0                                  [heap]
7f4df4252000-7f4df43ec000 r-xp 00000000 00:25 23161                      /lib64/libc-2.22.so
7f4df43ec000-7f4df45eb000 ---p 0019a000 00:25 23161                      /lib64/libc-2.22.so
7f4df45eb000-7f4df45ef000 r--p 00199000 00:25 23161                      /lib64/libc-2.22.so
7f4df45ef000-7f4df45f1000 rw-p 0019d000 00:25 23161                      /lib64/libc-2.22.so
7f4df45f1000-7f4df45f5000 rw-p 00000000 00:00 0
7f4df45f5000-7f4df45f7000 r-xp 00000000 00:25 23167                      /lib64/libdl-2.22.so
7f4df45f7000-7f4df47f7000 ---p 00002000 00:25 23167                      /lib64/libdl-2.22.so
7f4df47f7000-7f4df47f8000 r--p 00002000 00:25 23167                      /lib64/libdl-2.22.so
7f4df47f8000-7f4df47f9000 rw-p 00003000 00:25 23167                      /lib64/libdl-2.22.so
7f4df47f9000-7f4df4824000 r-xp 00000000 00:25 25974                      /lib64/libtinfo.so.5.9
7f4df4824000-7f4df4a23000 ---p 0002b000 00:25 25974                      /lib64/libtinfo.so.5.9
7f4df4a23000-7f4df4a27000 r--p 0002a000 00:25 25974                      /lib64/libtinfo.so.5.9
7f4df4a27000-7f4df4a2c000 rw-p 0002e000 00:25 25974                      /lib64/libtinfo.so.5.9
7f4df4a2c000-7f4df4a2d000 rw-p 00000000 00:00 0
7f4df4a2d000-7f4df4a6e000 r-xp 00000000 00:25 26789                      /lib64/libreadline.so.6.3

liked this article?

  • only together we can create a truly free world
  • plz support dwaves to keep it up & running!
  • (yes the info on the internet is (mostly) free but beer is still not free (still have to work on that))
  • really really hate advertisement
  • contribute: whenever a solution was found, blog about it for others to find!
  • talk about, recommend & link to this blog and articles
  • thanks to all who contribute!
admin