Which filesystem operations in HDFS is atomic?

Atomicity is a very important and fundamental property aspect of filesystems. Applications semantics and many functions depend on and only be available based on the atomicity models of the underlying filesystem. Which filesystem operations in HDFS is atomic? So that locks can be implemented on top of it. In a reasonably widely usable filesystem, some operations are expected to be atomic because they are often used to implement locking/exclusive access between processes using the filesystem. For HDFS, the processes are usually distributed in a cluster of servers instead of within a single OS.

As of Hadoop 3.2.1, the atomicity operations from a compatible filesystem (HDFS is one) are as follows. Most other operations come with no requirements or guarantees of atomicity.

– Creating a file. If the overwrite parameter is false, the check and creation MUST be atomic.
– Deleting a file.
– Renaming a file.
– Renaming a directory.
– Creating a single directory with mkdir().

Recursive directory deletion MAY be atomic. Although HDFS offers atomic recursive directory deletion, none of the other Hadoop FileSystems offer such a guarantee (including local FileSystems).