reading: coding-for-ssd & implement a kv

1.2.1   ssd 架构

• 接口主要是 Serial ATA (SATA), PCI Express (PCIe), 现在还有新的SAS.
• 通常都有RAM存储(256M+), 用作buffer/cache/map关系等

厂商给的数字通常是在某种条件下测得的, 不见得能表示真实性能:

In his articles about common flaws in SSD benchmarking [66], Marc Bevand mentioned that for instance it is common for the IOPS of random write workloads to be reported without any mention of the span of the LBA, and that many IOPS are also reported for queue depth of 1 instead of the maximum value for the drive being tested. There are also many cases of bugs and misuses of the benchmarking tools.

Correctly assessing the performance of SSDs is not an easy task.

1. span

2. queue-depth很重要.

3. 测试时间, 和SSD的size有很大关系:

   the performance of SSDs only drops under a sustained workload of random writes, which depending on the total size of the SSD can take just 30 minutes or up to three hours
   

   如下图, 30min后性能有明显下降:

   

   原因:

   • garbage collection must erase blocks as write commands arrive, therefore competing with the foreground operations from the host,
   • GC影响正常流量.

这里: http://www.storagereview.com/samsung_ssd_840_pro_review

1.2.2   如何做benchmark:

The parameters used are generally the following: - The type of workload: can be a specific benchmark based on data collected from users, or just only sequential or random accesses of the same type (ex: only random writes) - The percentages of reads and writes performed concurrently (ex: 30% reads and 70% writes) - The queue length: this is the number of concurrent execution threads running commands on a drive - The size of the data chunks being accessed (4 KB, 8 KB, etc.)

结果: - Throughput (对顺序读写) - ipos (对随机读写) - Latancy

1.3.1   基本操作

1. Reads are aligned on page size (It is not possible to read less than one page at once.)

   • 和hdd其实一样.

2. Writes are aligned on page size

   • 每次读一个page比较好理解, 每次怎么写一个page??

   • write amplification(写放大)

   • 写操作不是直接覆盖原来的page, 而是 拷贝这个page到ssd的RAM中, 在RAM中修改, 最后写到一个新的page里面去.

     • When data is changed, the content of the page is copied into an internal register, the data is updated, and the new version is stored in a “free” page, an operation called “read-modify-write”. The data is not updated in-place, as the “free” page is a different page than the page that originally contained the data. Once the data is persisted to the drive, the original page is marked as being “stale”, and will remain as such until it is erased.

3. Erases are aligned on block size

   • 对于write造成的 stale 状态的page, 需要一个erase操作, 变成 free 状态.
   • 但是, 又不能Erase单独的一个Page, 只能Erase整个block.
   • 这些都是由ssd控制器的garbage collection process来处理的, 所以这里有很多各种算法(因为寿命有限, 所以寿命管理也需要复杂算法)

1.3.2   写放大

写一个字节也会导致整个page的read-modify-write

应该尽量避免 small write.

1.3.3   wear leveling

写均衡. 避免总是写同一个page.

1.3.4   Flash Translation Layer (FTL)

Flash Translation Layer 是一个地址映射机制, 提供像磁盘一样的逻辑地址.

1. logical block mapping

   • 一个逻辑地址到page的有映射表.

   • This mapping table is stored in the RAM of the SSD for speed of access, and is persisted in flash memory in case of power failure.

   • When the SSD powers up, the table is read from the persisted version and reconstructed into the RAM of the SSD

   • block-level mapping or page-level mapping

     • page-level 太费内存. Let’s assume that an SSD drive has 256 pages per block. This means that block-level mapping requires 256 times less memory than page-level mapping

     • block-level 让写放大更加明显

     • log-block mapping

       • which uses an approach similar to log-structured file systems. Incoming write operations are written sequentially to log blocks. When a log block is full, it is merged with the data block associated to the same logical block number (LBN) into a free block
       • log-block mapping 应该是现在ssd写能达到和读一样性能的关键点.
       • This allows random writes to be handled like sequential writes.
       • 但是read requests need to check both the log-block mapping table and the data-block mapping table, 性能损耗应该很小.

2. garbage collection.

做ssd控制器的厂商比较少:

seven companies are providing controllers for 90% of the solid-state drive market(做ssd的厂商很多, 但是做控制芯片/算法的很少)

Erase: 1500-3500 μs Write: 250-1500 μs

A less important reason for blocks to be moved is the read disturb. Reading can change the state of nearby cells, thus blocks need to be moved around after a certain number of reads have been reached [14].

1.4.1   TRIM 命令

which can be sent by the operating system to notify the SSD controller that pages are no longer in use in the logical space

帮助控制器更好的GC.

The TRIM command will only work if the SSD controller, the operating system, and the filesystem are supporting it.

Under Linux, support for the ATA TRIM was added in version 2.6.33 . ext2 and ext3 filesystems do not support TRIM, ext4 and XFS , among others, do support it.

1.4.2   over-provisioning

Over-provisioning is simply having more physical blocks than logical blocks, by keeping a ratio of the physical blocks reserved for the controller and not visible to the user. Most manufacturers of professional SSDs already include some over-provisioning, generally in the order of 7 to 25%

1.4.3   Secure Erase

Some SSD controllers offer the ATA Secure Erase functionality

1.4.4   Internal Parallelism in SSDs

Due to physical limitations, an asynchronous NAND-flash I/O bus cannot provide more than 32-40 MB/s of bandwidth [5].

所以实际上, ssd控制器内部有多个芯片, 类似raid那样.

1.5.1   访问模式

在ssd里面, 因为映射关系是动态的, contiguous addresses in the logical space may refer to addresses that are not contiguous in the physical space.

1. 避免随机small write.

   • amall write 实际上需要写整个block. 所以写一个小数据和一个大数据都需要同样的copy-erase-write操作
   • 需要大量操作映射关系表.

2. Reads are faster than writes

   • 顺序读比随机读throughput好.

3. 其它优化

   • 对齐读写(有比较小的提升)
   • 打开 TRIM
   • IO scheduler, (默认是CFQ, NOOP or Deadline 会好一些)
   • 尽量不要用ssd存临时文件或做swap (写次数有限)