I want to kick off a user such as “bob” on a Linux box which I managed remotely. How to do this? bob may log on the Linux box by various ways: Log on the screen on the Linux box (locally) with a X server. Log on remotely by a ssh shell. Log on by
Read more
Tag: Linux
How to add a new HDFS NameNode metadata directory to an existing cluster?
Posted onWe have a running HDFS cluster. Currently, the NameNode metadata data directory has only one directory configured in hdfs-site.xml: <property> <name>dfs.namenode.name.dir</name> <value>file:///home/hadoop/hdfs/</value> <description>NameNode directory for namespace and transaction logs storage.</description> </property> We would like to add a new directory for dfs.namenode.name.dir to make replicas of the metadata on a separated disk for higher data reliability.
Read more
How to resize a batch of images on Linux?
Posted onResizing images on Linux with gThumb is easy. However, I have a batch of images inside a folder. Manually resizing them will consume too much time. How to automatically resize them on Linux with a script? You can use convert from ImageMagick together with bash script to resize images inside a directory. mkdir resize; IFS=$(echo
Read more
How to suppress “Entering/Leaving…” messages when invoking make recursively?
Posted onThe “Entering/Leaving…” messages when invoking another make by a make is kind of annoying. Is it possible to suppress these messages and how to suppress them? For GNU make, it is controlled by options: -w, –print-directory Print a message containing the working directory before and after other processing. This may be useful for tracking down
Read more
Fixing “Remote Host Identification Has Changed” Error When SSH to a Server
Posted onIf you encounter an error message like the one below when attempting to SSH to a server: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has
Read more
How to add a “status bar” to screen on Linux?
Posted onI noticed that some guys’ screen console has a status bar with tab numbers. That will be very useful for 1) know you are using screen rather than a normal terminal. 2) which tab you are working in. Below is my ~/.screenrc: hardstatus alwayslastline hardstatus string ‘%{= kG}[ %{G}%H %{g}][%= %{=kw}%?%-Lw%?%{r}(%{W}%n*%f%t%?(%u)%?%{r})%{w}%?%+Lw%?%?%= %{g}][%{B}%Y-%m-%d %{W}%c %{g}]’ It
Read more
How to detect whether an image is almost blank on Linux?
Posted onHow to detect whether a .jpg image is almost blank/black on Linux? Here is a bash function that I used to detect whether an image is black on Linux: function isBlank () { mean=`convert $1 -format “%[mean]” info:` echo “$mean” } It uses ImageMagic to generate formatted image characteristics (the mean). If the mean is
Read more
How to check the replication factor of a file in HDFS?
Posted onA related question: how to find the replication factors of files in a HDFS cluster? method 1: You can use the HDFS command line to ls the file. The second column of the output will show the replication factor of the file. For example, $ hdfs dfs -ls /usr/GroupStorage/data1/out.txt -rw-r–r– 3 hadoop zma 11906625598 2014-10-22
Read more
How to merge multiple jpg images to a pdf on Linux?
Posted onI have multiple jpg images as files like 001.jpg, 002.jpg … How to merge multiple jpg images to a pdf on Linux? convert is your good friend: convert *.jpg output.pdf
how to remove specific directories recursively
Posted onHow to remove .svn directories under hlfs dir recursively as follows. weiwei@weiwei-HP-Compaq-dx6128-MT-PX478AV:~/workshop1/hlfs > find ./ -name “.svn” ./test/build/.svn ./test/.svn ./output/conf/.svn ./output/lib32/.svn ./patches/.svn ./src/include/.svn ./src/include/api/.svn ./src/snapshot/.svn ./src/snapshot/unittest/build/.svn ./src/snapshot/unittest/.svn ./src/utils/.svn ./src/clean/Mapreducer/build/.svn ./src/clean/Mapreducer/.svn ./src/clean/.svn ./src/clean/unittest/.svn ./src/icache/.svn ./src/icache/unittest/.svn ./src/backend/.svn ./src/storage/.svn ./src/cache/.svn ./src/cache/unittest/.svn ./src/clone/.svn ./src/tools/.svn ./src/tools/unittest/.svn ./src/logger/.svn weiwei@weiwei-HP-Compaq-dx6128-MT-PX478AV:~/workshop1/hlfs > find ./ -name “.svn” | xargs rm -rf
How to download a rtmp stream on Linux?
Posted onHow to download a rtmp video stream on Linux? You can use mplayer to dump the rtmp stream like: mplayer -dumpstream rtmp://example.com/path/to/stream.mp4 It will generate ./stream.dump and you can rename it to the file with the extension you need like stream.mp4. The rtmp link usually can be found from the HTML or JavaScript source code
Read more
How to change an running HDFS cluster’s replication factor?
Posted onNow, I have a running HDFS cluster storing lost files. I want to change its default replication factor. How to change it? What will happen after it is changed? For example, I change from 2 to 3. Will HDFS automatically re-replicate the data chunks? First, the replication factor is client decided. Second, the replication factor
Read more
How to install latest version of Calibre?
Posted onHow to install latest version of Calibre? The version from my distro (Ubuntu, Linux Mint, Fedora) seem at 1.xx while the latest Calibre is already at 2.x. You may check Caibre website’s instruction: http://calibre-ebook.com/download_linux sudo -v && wget -nv -O- https://raw.githubusercontent.com/kovidgoyal/calibre/master/setup/linux-installer.py | sudo python -c “import sys; main=lambda:sys.stderr.write(‘Download failedn’); exec(sys.stdin.read()); main()”
How to install gitbook?
Posted onHow to install gitbook on my own Linux box? First, install node.js following https://www.systutorials.com/qa/1268/how-to-install-node-js-on-fedora or How to install node.js on Ubuntu/Linux Mint depending on your distro. Second, install gitbook by npm to /opt/: # cd /opt/ # npm install gitbook Then, the gitbook can be invoked by /opt/node_modules/gitbook/bin/gitbook.js You may need to install the latest
Read more
How to balance DataNode storage in HDFS?
Posted onAs nodes are added and deleted in a Hadoop cluster. Storage usage across DataNodes may be different. Some DataNodes’ disks are almost used up while some others’ are almost empty. How to balance data across DataNodes in HDFS? Hadoop provides the balancer to redistribute the data. Brief introduction to balancer in Hadoop: balancer. The design
Read more
How to find which package can be installed for a file, like “yum provides”?
Posted onHow to find which package can be installed for a file, like “yum provides”? That is, the package is not installed yet and I do not know the package for a file that I want. The apt-file tool can do the similar things as yum provides. You may need to install it first by sudo
Read more
How to install node.js on Ubuntu/Linux Mint?
Posted onHow to install node.js on Ubuntu/Linux Mint? This is how I install node.js on Linux Mint: # aptitude install nodejs nodejs-legacy npm The nodejs-legacy makes sure the command node will invoke node.js.
How to run gitbook on a headless server (make Calibre run in headless server)?
Posted onWhen use gitbook to generate ebook, Calibre reports this: RuntimeError: X server required. If you are running on a headless machine, use xvfb After xvfb is installed, it does not work either. How to make gitbook/Calibre work on a headless server? You need to wrap the command ebook-convert with xvfb-run. However, in gitbook (lib/generate/ebook/index.js), ebook-convert
Read more
How to find the DataNodes that actually store a file in HDFS?
Posted onA file may be splitted to many chunks and replications stored on many datanodes in HDFS. Now, the question is how to find the DataNodes that actually store a file in HDFS? You may use the dfsadmin -fsck tool from the Hadoop hdfs util. Here is an example: $ hadoop fsck /user/aaa/file.name -files -locations -blocks
Read more
How to config network in host (wireless network) for QEMU guest os
Posted onThe host os is connected into network by wireless one so how to let its guest os connect network. Take [1] as a reference. I run it successfully with following steps. 1, create /etc/qemu-ifup script and chmod it. 2, start a qemu guest os with command sudo ./qemu/qemu-system-x86_64 -enable-kvm -m 1024 -drive file=marss_dram.qcow2 -vnc 127.0.0.1:0
Read more