HDAC6, a class IIB HDAC isoenzyme, stands unique in its structural and physiological functions. Besides histone modification, largely due to its cytoplasmic localization, HDAC6 also targets several non-histone proteins including Hsp90, α-tubulin, cortactin, HSF1, etc. Thus, it is one of the key regulators of different physiological and pathological disease conditions. HDAC6 is involved in different signaling pathways associated with several neurological disorders, various cancers at early and advanced stage, rare diseases and immunological conditions. Therefore, targeting HDAC6 has been found to be effective for various therapeutic purposes in recent years. Though several HDAC6 inhibitors (HDAC6is) have been developed till date, only two ACY-1215 (ricolinostat) and ACY-241 (citarinostat) are in the clinical trials. A lot of work is still needed to pinpoint strictly selective as well as potent HDAC6i. Considering the recent crystal structure of HDAC6, novel HDAC6is of significant therapeutic value can be designed. Notably, the canonical pharmacophore features of HDAC6is consist of a zinc binding group (ZBG), a linker function and a cap group. Significant modifications of cap function may lead to achieve better selectivity of the inhibitors. This review details the study about the structural biology of HDAC6, the physiological and pathological role of HDAC6 in several disease states and the detailed structure-activity relationships (SARs) of the known HDAC6is. This detailed review will provide key insights to design novel and highly effective HDAC6i in the future.