On my Fedora 20, I find that the system automatically clean up file under /tmp/. This is convenient. However, it cause some problems for some programs. For example, HDFS puts its DataNode pid file under /tmp/ by default like hadoop-hadoop-datanode.pid. After it is cleaned up, the hadoop-daemon.sh script will consider there is no DataNode running.
Read more
Tag: Bash
How to encode spaces in curl GET request URL on Linux?
Posted onThe problem is like this: I would like to send a GET request using curl and the content has spaces, such as curl “http://example.com/send?msg=hello world” The space and the “world” will be left away if this command is executed directory on Linux. How to encode spaces in curl request URL? You can use the –date-urlencode
Read more
How to run screen on a Linux host reporting “Cannot make directory ‘/var/run/screen’: Permission denied”?
Posted onI want to run screen on a remote host on which I do not have root/sudo privilege and screen is not installed. I can compile or copy the screen program that can run on the host. However, it reports: “Cannot make directory ‘/var/run/screen’: Permission denied” As screen is not installed on the host, the directory
Read more
How to make curl request pass the Squid proxy?
Posted onI use curl to send requests to remote servers like following command. curl http://example.com/send –date-urlencode “data=hello” However, it is blocked by the network proxy squid in some networks with message like (some info anonymized with “…”): ERROR The requested URL could not be retrieved POST /… HTTP/1.1 Proxy-Authorization: Basic … User-Agent: Mozilla/4.0 (compatible;) Host: …
Read more
How to exclude last N columns in Bash on Linux?
Posted onHow to exclude last N columns of a string in bash? The point is that the number of columns in each line is uncertain (but > N). For example, I would like to remove the last 2 columns separated by ‘.’ in the following lines. systemd.3.gz systemd.mount.3.gz systemd.mount.f.3.gz The simple cut command cut -d’.’ -f1
Read more
How to sort all files recursively by modification time in a directory on Linux?
Posted onHow to sort all the files in a directory and subdirectories recursively by modification time on Linux? You can make use of find (GNU find, or plus stat), sort and sed together to achieve recursively sort files by modification time: find . -type f -printf ‘%T@ %pn’ | sort -k 1 -n | sed ‘s/^[^
Read more
How to limit the network rate used by scp in Linux?
Posted onI am using a shared network and scp to upload files. I do not want to use most of the bandwidth available. How to limit the network rate used by scp in Linux? You can use the -l option of scp to limit the rate of bandwidth. -l limit Limits the used bandwidth, specified in
Read more
How to resolve a domain name from a specific DNS server instead of the configured one in Linux?
Posted onI would like to check whether the domain name’s IP has changed in the authoritative DNS server. The system’s configured DNS server may still cache the old IP, which is fine. The question is how to query from the authoritative DNS server the new IP? You can use dig – DNS lookup utility to query
Read more
Un-exporting an Exported Variable in Bash in Linux
Posted onHow to un-export an imported variable in Bash on Linux? For example, first, I export a variable MODE like export MODE=debug but I want to unexport the MODE (not turn to be `””`, should be truly non-defined). You can export -n MODE or unset MODE For your reference (from the [`bash` manual](https://www.systutorials.com/docs/linux/man/1-bash/)): export [-fn] [name[=word]]
Read more
How to get the directory path and file name from a absolute path in Bash on Linux?
Posted onHow to get the directory path and file name from a absolute path in Bash on Linux? For example, with “/foo/bar/baz.txt”, it will produce: “/foo/bar/” and “baz.txt”. You also have the basename and dirname commands besides of the basename and dirname C API in Linux: [zma@laptop:~]$ p=”/foo/bar/baz.txt” [zma@laptop:~]$ dirname $p /foo/bar [zma@laptop:~]$ basename $p baz.txt
How to redirect STDOUT of sudo command on Linux and write to file as root?
Posted onA try like the following command failed $ sudo echo “echo hello” > /usr/local/bin/hello with an message bash: /usr/local/bin/hello: Permission denied It seems the writing to the file is executed under the non-root user. How to redirect STDOUT of sudo command on Linux and write the content to the file (/usr/local/bin/hello here) as root? You
Read more
How to convert all text from upper case to lower case on Linux?
Posted onHow to convert all text in a file from upper case to lower case and vice versa on Linux? Convert from upper case to lower case: tr ‘[:upper:]’ ‘[:lower:]’ < input.txt > output.txt Convert from lower case to upper case: tr ‘[:lower:]’ ‘[:upper:]’ < input.txt > output.txt
Good tutorials for screen on Linux
Posted onAny suggestions on good tutorials for screen on Linux? To start with screen: A quick tutorial on screen. After can use the basic functions of screen, you may check out A dummies introduction to GNU Screen and A guide to GNU Screen. During all the progresses, frequently check the screen man page.
How to email admins automatically after a Linux server starts?
Posted onManaging a cluster of servers, I would like to notified when a server is started. How to make the Linux servers email me or other admins automatically after they are started? I did this by adding a crontab entry on each servers like @reboot date | mailx -S smtp=smtp://smtp.example.com -s “`hostname` started” -r zma@example.com zma@example.com
Read more
How to merge multiple PDF files to a PDF on Linux?
Posted onconvert seems works not very well when merging PDFs. The quality is low. Any other better methods to merge multiple PDF files to a single PDF on Linux? ghostscript works the best for me on merging PDFs: gs -q -sPAPERSIZE=letter -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -dPDFSETTINGS=/prepress -sOutputFile=out.pdf in1.pdf in2.pdf in3.pdf merges in{1..3}.pdf to out.pdf.
How to avoid mounting failures blocking Linux booting?
Posted onSome entries in /etc/fstab may not not critical for booting Linux or even not available until Linux has booted. How to avoid the failures or unavailability of some mounting entries in /etc/fstab blocking the boot process of Linux? Please check the tutorial at Controlling Filesystem Mounting on Linux by Playing with /etc/fstab: Allow non-root users
Read more
How to set replication factors for HDFS directories?
Posted onIs it possible to set the replication factor for specific directory in HDFS to be one that is different from the default replication factor? This should set the existing files’ replication factors but also new files created in the specific directory. This can simplify the administration. We can set the replication factor of /tmp/ to
Read more
How to get files without certain strings in their files names (reverse of *string*) on Linux?
Posted onTo get files with certain string in their file names, it is quite straightforward: ls *string* However, how to do the reverse one: how to get files without certain strings in their files names on Linux? You can get a list of file names by a combination of find and xargs as follows: find .
Read more
Deleting a Specific Line From a Text File in Command Line in Linux
Posted onOn Linux, how to delete a specific line from a text file in command line? For example, to delete the 4th line from a file aaa bbb ccc ddd eee ffffff You can use the “stream editor for filtering and transforming text” sed. With GNU sed: sed -i ‘4d’ ./file Here, -i means edit the
Read more
How to find the number of files in each directories on Linux?
Posted onHow to find the number of files in each sub-directories of a directory on Linux? For example, . ├── a19 ├── a8 ├── d2 ├── ecfddd └── t1 The number of sub-directories can be quite large. How to find the number of files in each sub-directories here? You can use this piece of script to
Read more