Reading an entire binary file into memory can be accomplished via Files.readAllBytes():
byte[] bytes = Files.readAllBytes(binaryFile);
A similar method exists in the InputStream class as well.
While these methods are very convenient for relatively small files, they are not a good choice for large files. Trying to fetch large files into memory is prone to OOM errors and, obviously, will consume a lot of memory. Alternatively, in the case of huge files (e.g., 200 GB), we can focus on memory-mapped files (MappedByteBuffer). MappedByteBuffer allows us to create and modify huge files and treat them as a very big array. They look like they are in memory even if they are not. Everything happens at the native level:
try (FileChannel fileChannel = (FileChannel.open(binaryFile,
EnumSet.of(StandardOpenOption.READ)))) {
MappedByteBuffer mbBuffer = fileChannel.map(
FileChannel.MapMode.READ_ONLY, 0, fileChannel.size());
System.out.println(" Read: " + mbBuffer.limit() + " bytes");
}
For huge files it is advisable to traverse the buffer with a fixed size as follows:
private static final int MAP_SIZE = 5242880; // 5 MB in bytes
try (FileChannel fileChannel = FileChannel.open(
binaryFile, StandardOpenOption.READ)) {
int position = 0;
long length = fileChannel.size();
while (position < length) {
long remaining = length - position;
int bytestomap = (int) Math.min(MAP_SIZE, remaining);
MappedByteBuffer mbBuffer = fileChannel.map(
MapMode.READ_ONLY, position, bytestomap);
... // do something with the current buffer
position += bytestomap;
}
}