Assorted tools for interacting with servers

Personal list of commands to remember when running analyses on servers. This post is expected to grow from time to time as new commands/command-combinations are found to be useful.

# login with ssh
ssh -l user ip # e.g., myUser 186.333.444.111

# create a virtual screen
screen -S screenName # use something identifying the analysis in screenName

# back to virtual screen called "screenName"
screen -r screenName

# list screens
screen -r

# detach virtual screen keeping the analysis
Ctrl + a + d

# detach virtual screen AND kill the job
Ctrl + c

# copy files through ssh to server
# supports the -r recursive tag for several files/directories
scp user@ip:/path/to/files path/to/local/directory # from directory in user at ip to local directory 
scp path/to/local/file user@ip:/destination/directory
scp -r path/to/local/directory user@ip:/destination/directory

# monitor processes while they are running based on the filetype and modification time
find . -name *.log -ls | grep "date-right-now" # e.g., "Apr  06" or "Apr 21", note the space in the former
Advertisements

Searching for text content into GNU/Linux files (and possibly OSX too)

Machines running GNU/Linux are very powerful tools once we are used to the command line. As an example, let us imagine that we are looking for a file named importantScript.py, a text file containing python code that we forgot where we put but are sure about its name. The standard file search engine called catfish is the tool for looking for files by filename and path in the hard drive. It is similar to search engines in Windows and OSX, possibly with the difference that apparently Finder in OSX can look for keywords into the files (the latter property of Finder is according to my wife, I honestly stopped using OSX long long ago).

But, how about keywords inside files? For instance, we were working on a project with a lot of files written in \LaTeX with the knitr package, so that we can embed R code to execute during document compilation. In just one from a whole bunch of text files we used a special package but forgot how to use it again (it was useful but we just needed it once so far), and internet fell so that we can not rely on good ol’ google in order to look for the usage. For reasons that don’t matter for now (probably ignorance) we also overlook the local documentation. Are we lost? Probably not.

Let us assume we remember a keyword, maybe a package name, or another sort of keyword, for instance a macro (\mymacro in \LaTeX). We can use the command grep with the -r option in order to visit all of the files and subdirectories in the current directory looking for the keyword(s) of interest. First we open an instance of the command line (e.g., bash) and navigate to the directory from where we want to search. Use of explicit or relative paths are also possible but I leave them to the reader for exploration. After calling grep, there will appear some output indicating file names and some text after a semicolon (:); this means that the command found our keywords in some of the files in the search path. On the other hand, if the terminal gives us the control again by showing the prompt ($), it failed to find the keywords inside any of the files and files-within-subdirectories of our current path. For instance:

user@machine:~$ cd ~/Documents/millionsOfScripts
# now let's see its content
# by the way, this directory is completely hypothetical!
user@machine:~$ ls
block  class  devices   fs          kernel  power
bus    dev    firmware  hypervisor  module
# search recursively the files into each of the directories show above for 
# the word 'subfigure'
user@machine:~$ grep -r subfigure
module/project1/scripts4/toCompile/blocks.Rnw:\begin{subfigure}

The code above indicates that our recursive search for the term ‘subfigure’ found a single file called blocks.Rnw in the subdirectory module/project1/scripts4/toCompile that contained inside such file a line with the text \begin{subfigure}. Then we can visit that file, open it, and see how did we use that tool in the past when working with \LaTeX. Pretty useful when we just have a bare idea of what we are looking for.

I found it useful for exactly that case when I did not remember how to use the subfigure \LaTeX package and did not want to search in google; I then just went to a directory where I had some indication that files using the package could be placed (e.g., my folder with beamer presentations or my PhD folder with some of my scholarship report files, all of them in \LaTeX) and then let grep look for them. Hope you find it useful and funny, it could save your life some day (as most of the bash commands, so learn to love the command line!)

PS: grep can take very very long to execute if your directory has a lot of files, subdirectories, and they are large.