In HDFS, files are split into blocks and replicated across multiple DataNodes for fault tolerance and parallelism. When you need to know exactly which DataNodes hold a specific file’s blocks, use hdfs fsck with the -locations and -blocks flags. Basic command hdfs fsck /path/to/file -files -locations -blocks This queries the NameNode and returns the block…
HDFS uses the dfs.replication property in hdfs-site.xml to set a global default replication factor for all blocks. However, you can override this on a per-file or per-directory basis using the hdfs dfs -setrep command — useful for frequently accessed “hot” files that need higher availability. Basic syntax hdfs dfs -setrep [-R] [-w] <numReplicas> <path> Setting…
When processing application logs (like Hadoop/log4j output), you often need to extract entries from a specific time window. This is especially common in automated routines that run periodically—for example, pulling the last 4 hours of logs every 4 hours via cron. Log format considerations Most application logs follow a consistent timestamp format. For example, log4j…
When you need to see what versions of a package are available in your Ubuntu repositories, there are several approaches depending on how much detail you need. Using apt-cache showpkg The most direct method is apt-cache showpkg, which lists all available versions along with their repository sources: apt-cache showpkg thunderbird This output shows version numbers,…
Package managers on Debian and Ubuntu systems maintain detailed logs of all installations, upgrades, and removals. These logs are essential for auditing system changes, troubleshooting dependency issues, or understanding what was installed and when. Log File Locations The main log files you’ll work with are: /var/log/dpkg.log — Low-level package management operations logged by dpkg /var/log/apt/history.log…
When managing LVM storage, you need to know which physical disks support each logical volume — especially for risk assessment when hardware fails. This guide shows you how to trace those relationships. Using lvdisplay to map extents The lvdisplay command with the –maps flag shows how logical extents map to physical volumes: lvdisplay –maps This…
awk is the most efficient approach for this task. It processes the file in a single pass and doesn’t load data into memory: awk ‘{ total += $5; count++ } END { print total/count }’ file.txt This assumes your numeric value is the 5th whitespace-delimited field. For a file like: Run 0: Time used: 22.235331711…
When debugging a program or understanding what it actually touches on your filesystem, you need to see which files it opens. Two tools do this well: strace traces system calls in real time, while lsof inspects a running process without the overhead. Using strace to capture file operations The simplest approach is to run strace…
To toggle between Simplified Chinese and Traditional Chinese input modes in ibus-libpinyin, press Ctrl+Shift+F. This shortcut has been the standard method since ibus-pinyin 1.3.7 and works across modern versions. Common ibus-libpinyin Shortcuts Beyond script conversion, these shortcuts handle daily input tasks: Shortcut Function Ctrl+Shift+F Toggle Simplified/Traditional Chinese Ctrl+. Toggle full-width/half-width punctuation Ctrl+, Toggle English/Chinese punctuation…
When network interfaces fail or behave unexpectedly, you need to inspect driver details, hardware state, and kernel messages. Here’s the practical toolkit for diagnosing network hardware and driver issues. Kernel Messages and Logs Start by checking kernel logs where driver initialization and errors are recorded: dmesg | grep -i network dmesg | grep -i driver…
The nl command is purpose-built for adding line numbers to text files: nl text-file By default, nl numbers all non-empty lines and skips blank lines. This behavior is controlled by the body section style, which defaults to t (text only). Common nl options Number all lines including blanks: nl -ba text-file Start numbering at a…
Domain registration pricing has commoditized significantly over the past few years. Most registrars cluster around similar price points, and permanent discounts are increasingly rare. Rather than chasing one-off coupons that expire or have restrictive terms, understanding how registrars price domains and what you actually need will serve you better. Current Market Rates As of 2026,…
You’ve accidentally broken a file and need to restore it from an earlier commit without touching anything else in your repository. Here’s how. Restore a file to a specific commit To restore my_file.name to its state at a specific commit: git checkout commit_sha1 my_file.name Replace commit_sha1 with the actual commit hash. You can find it…
You sometimes need to know whether your shell is running inside screen or tmux. The most straightforward approach is checking the TERM environment variable. Using TERM When screen starts a shell, it sets TERM=screen by default: [user@host ~]$ echo $TERM screen Compare this to a normal terminal session: [user@host ~]$ echo $TERM xterm-256color The TERM…
When you install Core via opam, the interface files (.mli) are stored in your OPAM switch directory. The exact path depends on your OCaml version and OPAM setup. Standard Location The .mli files for Core are located at: ~/.opam/$(ocaml -version | awk ‘{print $NF}’)/lib/core/ where the OCaml version is part of the switch path. For…
The Jane Street OPAM repository contains specialized OCaml packages and libraries maintained by Jane Street, including their popular Core library and related tools. You’ll need to add this repository to your OPAM configuration to access these packages. Method 1: Clone and Add Locally If you prefer to clone the repository first and add it as…
Vim’s regex engine is powerful but requires understanding its syntax rules, which differ significantly from standard POSIX extended regex. Whether you’re doing basic substitutions or complex text transformations, knowing how Vim handles regex patterns will save you hours of frustration. Vim Regex Basics Vim supports multiple regex engines, controlled by the magic setting. By default,…
Spawning background processes in Bash is straightforward with the & operator, but there are several nuances depending on your use case. Basic approach The simplest method is appending & to your command: for i in $(cat ./all-hosts); do ssh “$i” “ifconfig | grep Link” & done Each & launches the process in the background and…
If you’re seeing elevated TCP segment retransmission counts on Xen VMs, this typically signals network congestion, packet loss, or CPU contention. Here’s how to identify the root cause and fix it. Check Your Current Retransmission Rate Start by gathering baseline metrics: ss -s Look for the retransmit counters: TCP: established 45 tw 3 alloc 48…
Linux routing tables exist in kernel memory during runtime. To persist routes across reboots, you need to store them on disk and have them reloaded during system startup. The mechanism varies depending on your network management setup. systemd-networkd and netplan (modern systems) On contemporary distributions using systemd-networkd or netplan, define routes in YAML configuration files….