Although the 5 1/4 Inch Floppy Disk was regarded as "small" after the step down from 8 inch disks, demand for higher capacities met an increase in capabilities on the manufacturer side and a next iteration in storage format was made possible through higher mechanical precision drive mechanics and better magneto-chemical film coatings. Over some time in the early 1980s, a number of physical formats were competing to become the next mainstream removable storage medium. In Japan, Sony introduced its own take on the situation with a double sided, single density (67.5 tpi) flexible disk in a rigid 3.5 inch cartridge style case in 1981. For example Epson's "portable floppy" disk drive PF-10, companion to the PX-8 computer, relied on these disks. The format introduced what would ultimately become the norm for "floppy" disks - a rigid case in the style of a "caddy" with a sliding metal shutter to protect the delicate disk surface from touching, dirt and dust. Convenience features like a sliding read/write-enable indicator and the self-contained protective case - which eliminated the need for polyethylene (DuPont Tyvek) disk packaging sleeves or paper envelopes were a winning combination. Also, the fact that the disk was slightly longer than wide and a label that indicated how to hold the disk, in sum effectively prevented users from inserting the disk in a wrong orientation.
The Microfloppy Industry Committee adopted this format and soon after, in 1982/1983, presented a slightly improved design based on Sony's prior work. Initially, the disk was only single sided and in double density, with an IBM-compatible formatted capacity of 360KB laid out on 80 tracks (SS/DD). The double sided double density variant (DS/DD) followed around 1984 with 720KB of IBM-compatible formatted capacity. IBM had previously tried to establish its own 3.5" format but never managed to find a large audience. Asian drive vendors at the time were getting more market share and with additional momentum from the home computer market, the Japan-led 3.5" disk we know today won out. Commodore shipped the 1581 3.5" floppy drive briefly but had much greater success with the format in the Amiga. Atari's ST also relied on 3.5" disks - both companies using Double Density disks but breaking own territory with different on-disk logical formats, incompatible with IBM PCs.
While the home computer market benefited from a stable-format recording-medium over many years - comparing 3.5" disks with game console ROM-cartridges - the more business-centric IBM user base demanded earlier, what the rest of the computer world would inevitably demand soon after: more capacity. The floppy disk answer from IBM was the introduction of the 3.5" Micro Floppy High Definition disk, where the rotating media was coated with a different iron oxide, allowing higher capacities. While DD disks used "simple" magnetic iron oxide which read/write heads could flip at 665 Oe (Oersted), the HD medium used cobalt-doped iron oxide, requiring stronger heads, able to deliver 720-750 Oe. This made disks and drives slightly incompatible on a hardware level, but consumers tended to ignore that and regarded the HD disks as "high quality" disks, which in actual daily use much too often actually worked.
The 3.5 inch disk dominated in the computer market throughout the 1990s. A number of vendors tried to establish high capacity formats, but neither the 2.88MB "ED" disk ("extra high density") or other similar formats saw wide adoption. The next step in removable recording media was not a floppy disk but the advent of the CD-ROM and writable laser disk media. A step in-between were "superfloppy" high-capacity magnetic-only formats like the ZIP disk format or "floptical" recording media, where a combination of magnetic and optical recording is used to record on a disk similar in size to a traditional 3.5" disk.