目录1.知识回顾2.前置知识:fwrite函数2.对比示例代码的运行结果示例代码1示例代码2:去掉示例代码1中所有字符串中的\n示例代码3:注释掉示例代码2的close(1)3.分析示例代码,进一步引入C语言的缓冲区前置知识: 刷新缓冲区初步结论进一步分析从内核代码看默认情况下的write将数据先写入内核缓冲区,再落磁盘结论得出原因查看printf、fprintf和fwrite写入的缓冲区FILE结构体4.回顾exit和_exit5.缓冲区刷新策略C语言缓冲区刷新所有方法1.知识回顾之前在一些文章提到过C语言的缓冲区:24.【C语言】getchar和putchar的使用89.【C语言】文件操作(6)本文进一步解释2.前置知识:fwrite函数函数声明为:size_t fwrite(const void ptr[restrict .size * .nmemb], size_t size, size_t nmemb, FILE *restrict stream);fwrite以二进制形式对文件进行操作(把ptr所指向的数组中的数据写入到给定流stream中),不局限于文本文件,可以看到ptr是指向用于写入的元素数组的指针,其类型为constvoid,那么传参给fwrite涉及到ptr的类型转换size表示一个块的大小(单位是字节),nmemb表示要求写入的块的个数,那么一共需要写入的大小为nmemb*size,但fwrite返回的是实际写入的块数(不一定等于nmemb),可能ptr指向数组的大小是小于nmemb*size的)stream是FILE*类型的指针2.对比示例代码的运行结果示例代码1#include stdio.h #include string.h #include unistd.h int main() { const char* str1 hello printf\n; const char* str2 hello fprintf\n; const char* str3 hello fwrite\n; printf(%s,str1); // stdout - 1 fprintf(stdout, %s,str2); // stdout - 1 fwrite(str3, strlen(str3), 1, stdout); // fwrite, stdout - 1 close(1); return 0; }运行结果:正常打印所有字符串示例代码2:去掉示例代码1中所有字符串中的\n#include stdio.h #include string.h #include unistd.h int main() { const char* str1 hello printf; const char* str2 hello fprintf; const char* str3 hello fwrite; printf(%s,str1); // stdout - 1 fprintf(stdout, %s,str2); // stdout - 1 fwrite(str3, strlen(str3), 1, stdout); // fwrite, stdout - 1 close(1); return 0; }运行结果:什么都没有打印示例代码3:注释掉示例代码2的close(1)#include stdio.h #include string.h int main() { const char* str1 hello printf; const char* str2 hello fprintf; const char* str3 hello fwrite; printf(str1); // stdout - 1 fprintf(stdout, str2); // stdout - 1 fwrite(str3, strlen(str3), 1, stdout); // fwrite, stdout - 1 //close(1); return 0; }运行结果:正常打印3.分析示例代码,进一步引入C语言的缓冲区前置知识: 刷新缓冲区刷新缓冲区指的是将缓冲区的数据写入流中然后清空缓冲区,那么刷新到显示器指的是刷新缓冲区的数据到显示器初步结论之前讲过,进程退出之前会自动刷新缓冲区对于示例代码2,printf、fprintf和fwrite都向缓冲区中写入不含有\n字符串,那么缓冲区不会强制刷新到显示器上,执行close(1),接着进程退出之前执行自动刷新缓冲区的操作,但没有字符串被打印出来推出初步的结论:这个缓冲区一定不在操作系统内,是C语言自带的缓冲区,它不是操作系统(内核)的缓冲区可以反向思考: 如果缓冲区在操作系统内部,那么调用C语言文件函数,其内部使用write系统调用将字符串写入到操作系统的缓冲区,由于close是Linux的系统调用,那么close(1)时会将字符串刷新到屏幕,但示例代码2的现象是没有字符串被打印出来进一步分析通过printf、fprintf和fwrite写入字符串,字符串会先暂存到C语言提供的缓冲区,等到合适的时机时再通过write将它们写入到系统的缓冲区,且显示字符串需要刷新操作系统内部的缓冲区准确来说: 默认情况下(类似int fd open(testfile, O_WRONLY|O_CREAT, 0644);write(fd, hello, 5);),write先将数据写入到系统的缓冲区,待时机成熟后再将缓冲区的数据刷新到磁盘从内核代码看默认情况下的write将数据先写入内核缓冲区,再落磁盘以EXT4系统为例,看linux v6.19.10的write函数的调用链(读者可以去read_write.c - fs/read_write.c - Linux source code v6.19.10 - Bootlin Elixir Cross Referencer追踪验证)write系统调用→ksys_write→vfs_write→ext4的struct file(定义在/fs/ext4/file.c中)没有file-f_op-write→new_sync_write→ext4_file_write_iter→没有DAX和O_DIRECT标志的情况→ext4_buffered_write_iter→generic_perform_write→copy_folio_from_iter_atomic→......→generic_write_sync→......这里给出关键函数的定义:static ssize_t ext4_buffered_write_iter(struct kiocb *iocb, struct iov_iter *from) { ssize_t ret; struct inode *inode file_inode(iocb-ki_filp); if (iocb-ki_flags IOCB_NOWAIT) return -EOPNOTSUPP; inode_lock(inode); ret ext4_write_checks(iocb, from); if (ret 0) goto out; ret generic_perform_write(iocb, from); out: inode_unlock(inode); if (unlikely(ret 0)) return ret; return generic_write_sync(iocb, ret); }看两个函数的作用:先看ssize_t generic_perform_write(struct kiocb *iocb, struct iov_iter *i) { struct file *file iocb-ki_filp; loff_t pos iocb-ki_pos; struct address_space *mapping file-f_mapping; const struct address_space_operations *a_ops mapping-a_ops; size_t chunk mapping_max_folio_size(mapping); long status 0; ssize_t written 0; do { struct folio *folio; size_t offset; /* Offset into folio */ size_t bytes; /* Bytes to write to folio */ size_t copied; /* Bytes copied from user */ void *fsdata NULL; bytes iov_iter_count(i); retry: offset pos (chunk - 1); bytes min(chunk - offset, bytes); balance_dirty_pages_ratelimited(mapping); if (fatal_signal_pending(current)) { status -EINTR; break; } status a_ops-write_begin(iocb, mapping, pos, bytes, folio, fsdata); if (unlikely(status 0)) break; offset offset_in_folio(folio, pos); if (bytes folio_size(folio) - offset) bytes folio_size(folio) - offset; if (mapping_writably_mapped(mapping)) flush_dcache_folio(folio); /* * Faults here on mmap()s can recurse into arbitrary * filesystem code. Lots of locks are held that can * deadlock. Use an atomic copy to avoid deadlocking * in page fault handling. */ copied copy_folio_from_iter_atomic(folio, offset, bytes, i); flush_dcache_folio(folio); status a_ops-write_end(iocb, mapping, pos, bytes, copied, folio, fsdata); if (unlikely(status ! copied)) { iov_iter_revert(i, copied - max(status, 0L)); if (unlikely(status 0)) break; } cond_resched(); if (unlikely(status 0)) { /* * A short copy made -write_end() reject the * thing entirely. Might be memory poisoning * halfway through, might be a race with munmap, * might be severe memory pressure. */ if (chunk PAGE_SIZE) chunk / 2; if (copied) { bytes copied; goto retry; } /* * folio is now unlocked and faults on it can be * handled. Ensure forward progress by trying to * fault it in now. */ if (fault_in_iov_iter_readable(i, bytes) bytes) { status -EFAULT; break; } } else { pos status; written status; } } while (iov_iter_count(i)); if (!written) return status; iocb-ki_pos written; return written; } EXPORT_SYMBOL(generic_perform_write);generic_perform_write内部调用了write_begin函数:status a_ops-write_begin(iocb, mapping, pos, bytes,folio, fsdata);在/Documentation/filesystems/vfs.rst里面明确提到了write_begin函数的职责The filesystem must returnthe locked pagecache(页缓存! 说明了generic_perform_write负责将数据写入页缓存) foliofor thespecified offset:write_begin Called by the generic buffered write code to ask the filesystem to prepare to write len bytes at the given offset in the file. The address_space should check that the write will be able to complete, by allocating space if necessary and doing any other internal housekeeping. If the write will update parts of any basic-blocks on storage, then those blocks should be pre-read (if they havent been read already) so that the updated blocks can be written out properly. The filesystem must return the locked pagecache folio for the specified offset, in *foliop, for the caller to write into. It must be able to cope with short writes (where the length passed to write_begin is greater than the number of bytes copied into the folio). A void * may be returned in fsdata, which then gets passed into write_end. Returns 0 on success; 0 on failure (which is the error code), in which case write_end is not called.继续看generic_write_sync:/* * Sync the bytes written if this was a synchronous write. Expect ki_pos * to already be updated for the write, and will return either the amount * of bytes passed in, or an error if syncing the file failed. */ static inline ssize_t generic_write_sync(struct kiocb *iocb, ssize_t count) { if (iocb_is_dsync(iocb)) { int ret vfs_fsync_range(iocb-ki_filp, iocb-ki_pos - count, iocb-ki_pos - 1, (iocb-ki_flags IOCB_SYNC) ? 0 : 1); if (ret) return ret; } else if (iocb-ki_flags IOCB_DONTCACHE) { struct address_space *mapping iocb-ki_filp-f_mapping; filemap_flush_range(mapping, iocb-ki_pos - count, iocb-ki_pos - 1); } return count; } /** * vfs_fsync_range - helper to sync a range of data metadata to disk * file: file to sync * start: offset in bytes of the beginning of data range to sync * end: offset in bytes of the end of data range (inclusive) * datasync: perform only datasync * * Write back data in range start..end and metadata for file to disk. If * datasync is set only metadata needed to access modified file data is * written. */ int vfs_fsync_range(struct file *file, loff_t start, loff_t end, int datasync) { struct inode *inode file-f_mapping-host; if (!file-f_op-fsync) return -EINVAL; if (!datasync (inode_state_read_once(inode) I_DIRTY_TIME)) mark_inode_dirty_sync(inode); return file-f_op-fsync(file, start, end, datasync); } EXPORT_SYMBOL(vfs_fsync_range);generic_write_sync内部调用vfs_fsync_range,在vfs_fsync_range定义上方的注释写得很清楚:helper to sync a range of data metadata to disk,说明了generic_write_sync负责将数据同步到磁盘结论结论1: write内部调用generic_perform_write和generic_write_sync,其中 generic_perform_write负责将数据写入页缓存(Page Cache),generic_write_sync负责将数据同步到磁盘继而得出结论2: 默认情况下(类似int fd open(testfile, O_WRONLY|O_CREAT, 0644);write(fd, hello, 5); 不含DAX和O_DIRECT标志),write先将数据写入到内核的缓冲区(准确来说是页缓存!),待时机成熟后再将缓冲区的数据刷新到磁盘得出原因通过进一步分析,使用close(1)导致fd1的文件(stdout)被关闭,导致进程退出前自动刷新内核缓冲区时,无法将C语言提供的缓冲区中字符串的指针链接(不是拷贝,可以通过gdb的rwatch命令验证)到操作系统的缓冲区(因为找不到fd1对应的缓冲区)然后刷新操作系统的缓冲区那么可以在close(1)之前将C语言缓冲区的字符串写入到操作系统的缓冲区,可以使用write系统调用例如以下代码:#include stdio.h #include string.h #include unistd.h int main() { const char* str1 hello printf; const char* str2 hello fprintf; const char* str3 hello fwrite; write(1, str1, strlen(str1)); write(1, str2, strlen(str2)); write(1, str3, strlen(str3)); close(1); return 0; }即使在关闭文件描述符fd1后之前使用write写入的数据也已经确实发送到显示器了运行结果:或者这样改:#include stdio.h #include string.h #include unistd.h int main() { const char* str1 hello printf; const char* str2 hello fprintf; const char* str3 hello fwrite; printf(%s,str1); // stdout - 1 fprintf(stdout, %s,str2); // stdout - 1 fwrite(str3, strlen(str3), 1, stdout); // fwrite, stdout - 1 fflush(stdout); close(1); return 0; }运行结果:查看printf、fprintf和fwrite写入的缓冲区以含有fflush(stdout)的代码为例,先在这些地方下断点:r命令执行rprintf、fprintf和fwrite写入的缓冲区在stdout指向的FILE结构体中,gdb下可以这样查看stdout指向的FILE结构体:p *stdoutc命令继续执行c再次查看stdout指向的FILE结构体,发现已经写入了一个字符串:执行fprintf后,又写入了字符串:执行fwrite后,又写入了字符串:执行fflush(stdout),会将FILE结构体中的字符串的指针链接到操作系统的缓冲区中,而是否刷新到屏幕上是操作系统决定的,可以看C89文档的描述:fflush函数使得该流中任何未写入的数据被传送给主机环境,并写入文件对于fflush内部的实现,可以看glibc在libio/iofflush.c的源码结论: 用户刷新的本质,就是将数据通过fd1和write系统调用写入到内核(上面提到的主机环境)中(*注: 目前认为,只要将数据刷新到了内核,数据就到可以硬件了)FILE结构体由前面的分析可知: 文件操作绕不开FILE,FILE结构体里面含有打开文件的缓冲区字段和维护信息那么打开n个文件就有n个文件描述符,也就有n个语言级别上的缓冲区FILE结构体的源代码在glibc-2.42的libio/bits/types/FILE.h中给出了FILE的重定义:#ifndef __FILE_defined #define __FILE_defined 1 struct _IO_FILE; /* The opaque type of streams. This is the definition used elsewhere. */ typedef struct _IO_FILE FILE; #endif可以发现FILE是struct _IO_FILE的简写在glibc-2.42的libio/bits/types/struct_FILE.h中,给出了结构体完整的定义:/* The tag name of this struct is _IO_FILE to preserve historic C mangled names for functions taking FILE* arguments. That name should not be used in new code. */ struct _IO_FILE { int _flags; /* High-order word is _IO_MAGIC; rest is flags. */ /* The following pointers correspond to the C streambuf protocol. */ char *_IO_read_ptr; /* Current read pointer */ char *_IO_read_end; /* End of get area. */ char *_IO_read_base; /* Start of putbackget area. */ char *_IO_write_base; /* Start of put area. */ char *_IO_write_ptr; /* Current put pointer. */ char *_IO_write_end; /* End of put area. */ char *_IO_buf_base; /* Start of reserve area. */ char *_IO_buf_end; /* End of reserve area. */ /* The following fields are used to support backing up and undo. */ char *_IO_save_base; /* Pointer to start of non-current get area. */ char *_IO_backup_base; /* Pointer to first valid character of backup area */ char *_IO_save_end; /* Pointer to end of non-current get area. */ struct _IO_marker *_markers; struct _IO_FILE *_chain; int _fileno; int _flags2:24; /* Fallback buffer to use when malloc fails to allocate one. */ char _short_backupbuf[1]; __off_t _old_offset; /* This used to be _offset but its too small. */ /* 1column number of pbase(); 0 is unknown. */ unsigned short _cur_column; signed char _vtable_offset; char _shortbuf[1]; _IO_lock_t *_lock; #ifdef _IO_USE_OLD_IO_FILE };1.可以看到里面的int _fileno就是文件描述符,原因如下:因为IO相关函数与系统调用接口对应,并且库函数封装系统调用,所以本质上访问文件都是通过fd访问的,因此C库当中的FILE结构体内部必定封装了fd2.FILE对象属于用户(因为是C语言标准规定的,而且编程语言属于用户层),里面的缓冲区是用户的缓冲区3.每用C语言的fopen函数打开一个文件就要创建该文件的FILE对象4.回顾exit和_exit之前在OS23.【Linux】进程终止文章提到了exit和_exit,这里简单回顾:1. exit是C语言退出函数,访问C语言提供的缓冲区是合情合理的,那么使用exit退出时会刷新C语言的缓冲区到内核2. _exit是系统调用,作用也是退出,但和exit不一样的是: _exit处于底层,无法访问C语言提供的缓冲区,也就不会刷新C语言提供的缓冲区,_exit会close(1)然后结束进程5.缓冲区刷新策略C语言提供的缓冲区和操作系统内部的缓冲区的刷新策略有所不同,这里分开说C语言缓冲区刷新所有方法1.行缓冲: 直到见到\n才刷新,其余情况不刷新(例如显示器)2.全缓冲: 缓冲区满了才刷新(例如普通文件的写入)3.无缓冲: 直接刷新如果是打印到显示器,行刷新或程序结束的时候刷新如果是打印到文件,那么只有程序结束的时候才刷新冲刷缓冲区的策略可能丢弃数据也可能将数据写入文件