How to install latest version of Calibre?

Posted on

How to install latest version of Calibre? The version from my distro (Ubuntu, Linux Mint, Fedora) seem at 1.xx while the latest Calibre is already at 2.x. You may check Caibre website’s instruction: http://calibre-ebook.com/download_linux sudo -v && wget -nv -O- https://raw.githubusercontent.com/kovidgoyal/calibre/master/setup/linux-installer.py | sudo python -c “import sys; main=lambda:sys.stderr.write(‘Download failedn’); exec(sys.stdin.read()); main()”

How to find which package can be installed for a file, like “yum provides”?

Posted on

How to find which package can be installed for a file, like “yum provides”? That is, the package is not installed yet and I do not know the package for a file that I want. The apt-file tool can do the similar things as yum provides. You may need to install it first by sudo
Read more

How to install gitbook?

Posted on

How to install gitbook on my own Linux box? First, install node.js following https://www.systutorials.com/qa/1268/how-to-install-node-js-on-fedora or How to install node.js on Ubuntu/Linux Mint depending on your distro. Second, install gitbook by npm to /opt/: # cd /opt/ # npm install gitbook Then, the gitbook can be invoked by /opt/node_modules/gitbook/bin/gitbook.js You may need to install the latest
Read more

How to balance DataNode storage in HDFS?

Posted on

As nodes are added and deleted in a Hadoop cluster. Storage usage across DataNodes may be different. Some DataNodes’ disks are almost used up while some others’ are almost empty. How to balance data across DataNodes in HDFS? Hadoop provides the balancer to redistribute the data. Brief introduction to balancer in Hadoop: balancer. The design
Read more

How to run gitbook on a headless server (make Calibre run in headless server)?

Posted on

When use gitbook to generate ebook, Calibre reports this: RuntimeError: X server required. If you are running on a headless machine, use xvfb After xvfb is installed, it does not work either. How to make gitbook/Calibre work on a headless server? You need to wrap the command ebook-convert with xvfb-run. However, in gitbook (lib/generate/ebook/index.js), ebook-convert
Read more

How to config network in host (wireless network) for QEMU guest os

Posted on

The host os is connected into network by wireless one so how to let its guest os connect network. Take [1] as a reference. I run it successfully with following steps. 1, create /etc/qemu-ifup script and chmod it. 2, start a qemu guest os with command sudo ./qemu/qemu-system-x86_64 -enable-kvm -m 1024 -drive file=marss_dram.qcow2 -vnc 127.0.0.1:0
Read more

How to find the DataNodes that actually store a file in HDFS?

Posted on

A file may be splitted to many chunks and replications stored on many datanodes in HDFS. Now, the question is how to find the DataNodes that actually store a file in HDFS? You may use the dfsadmin -fsck tool from the Hadoop hdfs util. Here is an example: $ hadoop fsck /user/aaa/file.name -files -locations -blocks
Read more

How to write /etc/fstab entry for –bind mounting?

Posted on

How to write /etc/fstab entry for –bind mounting like mount –bind /home/hadoop/hdfs/store-tmp /home/store/tmp From man 8 mount: Since Linux 2.4.0 it is possible to remount part of the file hierarchy somewhere else. The call is mount –bind olddir newdir or shortoption mount -B olddir newdir or fstab entry is: /olddir /newdir none bind

How to increase the number of files allowed to be opened on Linux?

Posted on

On my system: $ ulimit -n 1024 Some tools like GATK are aggressive in creating temporary files by creating more than 1000 files under /tmp/. This will cause the program to fail. How to increase the number of files allowed to be opened on Linux? To increase the max number of open files to 10240,
Read more

How to change number of replications of certain files in HDFS?

Posted on

The HDFS has a configuration in hdfs-site.xml to set the global replication number of blocks with the “dfs.replication” property. However, there are some “hot” files that are access by many nodes. How to increase the number of blocks for these certain files in HDFS? You can the replication number of certain file to 10: hdfs
Read more

How to get logs of a specific time range on Linux?

Posted on

The logs I am processing is Hadoop log (log4j). It is in format like: 2014-09-20 21:55:11,855 INFO org.apache.hadoop.nfs.nfs3.IdUserGroup: Updated user map size: 36 2014-09-20 21:55:11,863 INFO org.apache.hadoop.nfs.nfs3.IdUserGroup: Updated group map size: 55 2014-09-20 22:10:11,907 INFO org.apache.hadoop.nfs.nfs3.IdUserGroup: Update cache now 2014-09-20 22:10:11,907 INFO org.apache.hadoop.nfs.nfs3.IdUserGroup: Not doing static UID/GID mapping because ‘/etc/nfs.map’ does not exist. Now, I
Read more

What’s the difference between Reliability, Durability, and Availability for data storage system?

Posted on

Some important concepts in distributed system like Hadoop distributed file system, Google file system and so on. Answer from http://www.quora.com/Whats-the-difference-between-Reliability-Durability-and-Availability-for-data-storage-system The difference between durability and availability is fairly simple. Durability is about what happens when all power goes out everywhere. Has all data been written to stable storage that doesn’t require power (e.g. disk/flash), in
Read more

Making Hadoop Java process heap larger?

Posted on

In Hadoop 2.5.0, I use ‘ps -aux’ and find the Java process has options: -Xmx1000m However, my nodes have 32GB memory. How to make Hadoop Java process heap larger? In yarn-env.sh, you can find: # For setting YARN specific HEAP sizes please use this # Parameter and set appropriately # YARN_HEAPSIZE=1000 In hadoop-env.sh, you can
Read more