How to use the Foreign Function API in Java 22 to Call C Libraries

This article explains how to call C libraries like fopen, fgets, and fclose from Java.

May 5, 2024

As mentioned later in this post, there’s almost no (java level) abstraction exposed to the user so if you want to use some c library and go on with your day, you’re better off using something built on the API or sticking with JNA till the API matures enough (or start a project to create bindings for the library)

Introduction

Before the release of Java 22, the usual way to call foreign functions was with the Java Native Interface (JNI). Anyone who has used (or tried to use) JNI would agree that it was very easy to shoot yourself in the foot and accidentally shoot someone else in the foot while the first shooting was still happening.

I’m not going to dive into how easy it is to mess up, but I will say this: To use JNI, you have to write C code. Let me rephrase that: for developers that have been pampered with garbage collection and safety to use JNI, they had to write C code. What could go wrong? There are other solutions built on JNI where you don’t have to write the C bindings manually, but … meh.

I’ll admit that JNI is very powerful. But what if we didn’t have to write C code, like the way it’s done in other languages? Enter the star of today’s writing: Foreign Function and Memory (FFM) API from the java.lang.foreign package.

This article will review how to read the contents of a file using fopen, fclose and fgets from the C standard library. This might get long and technical, so buckle up!

Using the FFM API to call regular functions that already exist in the Java standard library makes no sense because there will always be an overhead. The API is best used for libraries that cannot be rewritten in Java. I chose functions from the C standard library because it’s available to almost everyone.

Setting up LibC class and fopen

The Java language architects made the API as open as possible. Aside from the code that relates to foreign functions, there’s almost no abstraction. So, a little bit of setup is required to get going. My guess is that they didn’t want to constrain developers too much.

In a file named LibC.java:

import java.lang.foreign.*;

class LibC {
	private final Arena allocator;
	Linker linker = Linker.nativeLinker();
	SymbolLookup lib = linker.defaultLookup();

	LibC(Arena arena) {
		this.allocator = arena;
	}
}

Think of an Arena as an allocator. It is how memory is allocated and freed. If the arena is closed, all the memory allocated through it is freed. You’ll see more of it when we get the main method.
Linker.nativeLinker() simply … gets the native linker.
linker.defaultLookup() implicitly loads all the common libraries that are typically present on PCs. libc - the C standard library is one of them.

To set up fopen, add the following below SymbolLookup lib = linker.defaultLookup();:

// Required Imports
import static java.lang.foreign.ValueLayout.*;
import java.lang.invoke.MethodHandle;
// End imports

MemorySegment fopenAddress = lib.find("fopen").orElseThrow();
//FILE *fopen(const char *filename, const char *mode)
FunctionDescriptor fopenDesc =
		FunctionDescriptor.of(ADDRESS, ADDRESS, ADDRESS);
MethodHandle fopen = linker.downcallHandle(fopenAddress, fopenDesc);

The above code perfectly shows my love/hate relationship with Java. It’s verbose, lots of typing, I even had to skip the use of private final because it would get too long. But sometimes, it also makes sense. I mean … could it get any more descriptive than naming something FunctionDescriptor? Sure you could use FuncDescriptor or FuncDesc or be like go programmers and name it fd (please don’t fight me).

I don’t know about you but that looks scary. Let’s break it down:

MemorySegment
- Think of this as the equivalent of a C pointer. Anytime a C function returns a pointer, or takes in a pointer argument MemorySegment is used to represent it in java code.
lib.find("fopen").orElseThrow():
- Recall that lib contains all the common libraries that are typically found on a PC, and the C standard library is one of them.
- The code finds the address (pointer) of where fopen is stored, and returns it, or throws an exception if it wasn’t found.
FunctionDescriptor.of(ADDRESS, ADDRESS, ADDRESS):
- The code is exactly what it sounds like. It describes the signature of fopen
- The first argument is the return type of fopen - A memory address
- The 2nd and 3rd (and subsequent) arguments are the parameters. They’re all memory addresses.
- There’s also a FunctionDescriptor.ofVoid() that only allows parameters, and returns void
MethodHandle fopen = linker.downcallHandle(fopenAddress, fopenDesc)
- This basically creates a way to actually call the function.
- In general, it is used to call a foreign function in any language.

Any bindings you create will typically follow the same pattern: Find the library, describe the function, create a way to call it from java.

With that, we can create method to publicly access it. Below the LibC constructor:

MemorySegment fopen(String filePath, String mode) {
	MemorySegment pathPtr = allocator.allocateFrom(filePath);
	MemorySegment modePtr = allocator.allocateFrom(mode);

	try {
		return (MemorySegment) fopen.invoke(pathPtr, modePtr);
	}
	catch (Throwable e) {
		return MemorySegment.NULL;
	}
}

Note that type inference (kinda) has been around since Java 10. I’m explicitly using the types for clarity.

There are some new things, let’s address them:

MemorySegment pathPtr = allocator.allocateFrom(filePath):
- We’ve seen MemorySegment before. It’s a pointer.
- allocator.allocateFrom(filePath) creates a pointer from the file path string
- modePtr also follows the same pattern
return (MemorySegment) fopen.invoke(pathPtr, modePtr);
- Recall the signature (and FunctionDescriptor) of fopen: FILE *fopen(const char *filename, const char *mode). It returns a pointer, and takes in 2 pointers as its parameter.
- fopen.invoke() comes from the MethodHandle we created above. It invokes the foreign function, and returns an Object, so we need to cast to the described return type - an address i.e a MemorySegment
- There are several things that could go wrong, so .invoke() throws a Throwable - the super class of all errors and exceptions.
- Returning NULL is not the best way to handle an exception like this, but let’s keep it simple.

More on Arenas

As mentioned earlier, an arena is like an allocator.

There are some edge cases but most of the time, every memory you create in C has to be linked to an arena in Java. Even if you use malloc to allocate memory and get a MemorySegment back, before you use it, you have to link it to an arena, and in that same process of linking, you have to pass a Consumer that would free the memory segment. If you really need the memory without a Consumer to free it (not sure why), there’s a way to do that too.

Here’s a really simple example:

// Assume we've setup a LibC class that that DOESN'T take an 
// Arena as a constructor parameter. It contains both malloc and
// free methods. In a main method: 
var libc = new LibC();
MemorySegment mallocedMem = libc.malloc(20); // allocate 20 bytes

// outputs 0, we can't use it yet! An exception will be thrown 
// if we work with it.
System.out.println(mallocedMem.byteSize());
mallocedMem.getString(0); // IndexOutOfBoundsException

To be able to actually use it, we need something like this:

try (Arena allocator = Arena.ofConfined()) {
   // We could use the arena in the malloc call method but for
   // demonstration purposes, our library still doesn't have access to it.
   var libc = new LibCTest();
   
   // Size will be allocated later. For now, just get a reference
   MemorySegment mallocedMem = libc.malloc();
   System.out.println("Address: " + mallocedMem);
   
   Consumer<MemorySegment> cleanup = libc::free;
   
   // cleanup is called after the allocator is closed
   mallocedMem = mallocedMem.reinterpret(20, allocator, cleanup);
   // Same address as above, but with different size
   System.out.println("Address " + mallocedMem);
   System.out.println(mallocedMem.getString(0)); // prints empty string
}
// allocator closed here, mallocedMem is freed

I think it’s a nice way to make sure we can use C and not inherit the memory issues.

Main/Runner class

Create a file named file.txt and put some text in it. I’m not saying you should use this text, but it’ll be cool if you did:

What is that thing?
This is Appa, my flying bison.
Right, and this is Katara, my flying sister.

In a file named Runner.java:

import java.lang.foreign.*;
// we have to run with java 22 so we might as well use the 
// unnamed classes preview feature. Your IDE might yell at 
// you and ask to enable preview features.
void main() {
	try (Arena allocator = Arena.ofConfined()) {
		var libc = new LibC(allocator);

		MemorySegment filePtr = libc.fopen("file.txt", "r");
		if (filePtr.equals(MemorySegment.NULL)) {
			System.out.println("File could not be opened or invoke call failed");
			return;
		}
		System.out.println("File opened successfully");
	}
}

If you run with java --enable-preview --source 22 Runner.java You should see the success message or an error message if you change to a file that doesn’t exist … and some warnings. Those warnings are there to remind you that you are using methods that might cause the JVM to crash.

Adding fgets and fclose

As mentioned earlier, the steps to add more libraries are the same, so I won’t go over already discussed terminologies. Just after the MethodHandle for fopen, add the following:

//char *fgets(char *str, int n, FILE *stream)
MemorySegment fgetsAddress = lib.find("fgets").orElseThrow();
FunctionDescriptor fgetsDesc =
		FunctionDescriptor.of(ADDRESS, ADDRESS, JAVA_INT, ADDRESS);
MethodHandle fgets = linker.downcallHandle(fgetsAddress, fgetsDesc);

//int fclose(FILE *stream)
MemorySegment fcloseAddress = lib.find("fclose").orElseThrow();
FunctionDescriptor fcloseDesc =
		FunctionDescriptor.of(JAVA_INT, ADDRESS);
MethodHandle fclose = linker.downcallHandle(fcloseAddress, fcloseDesc);

The only unfamiliar code above should be the FunctionDescriptor. For fgetsDesc, we’re saying fgets returns an (address) pointer, takes a pointer, an int, and another pointer.

For fcloseDesc, we’re saying fclose returns an int, and takes an address

To be safer, you could use interfaces to differentiate between a C FILE* and a generic MemorySegment

And for the methods to publicly access them, add the following below MemorySegment fopen(String, String):

MemorySegment fgets(MemorySegment buffer, int size, MemorySegment filePtr) {
	try {
		return (MemorySegment) fgets.invoke(buffer, size, filePtr);
	}
	catch (Throwable e) {
		return MemorySegment.NULL;
	}
}

int fclose(MemorySegment filePtr) {
	try {
		return (int) fclose.invoke(filePtr);
	}
	catch (Throwable e) {
		return -1;
	}
}

Nothing unfamiliar here either. In fclose, instead of casting to a MemorySegment, we cast to an int because in the FunctionDescriptor, we said we expect an int as a result.

And finally, in the main method, replace System.out.println("File opened ... with:

MemorySegment buffer = allocator.allocate(100); // allocate 100 bytes
MemorySegment result;
for (; ; ) {
   result = libc.fgets(buffer, 100, filePtr);
   if (result.equals(MemorySegment.NULL)) break;
   
   System.out.print(buffer.getString(0));
}

if (libc.fclose(filePtr) != 0) {
   System.out.println("File was not closed");
}

Again, nothing too unfamiliar here either. It’s like a java flavoured version of how you read a file in plain C. The buffer.getString returns the string stored in the buffer at offset 0. So, if we wanted to print from the 5th character, we would use 5 instead.

If you run again with java --enable-preview --source 22 Runner.java, you should see the file contents printed out (and the warnings).

Wrapping up

What are your thoughts on this compared to JNI? It’s a lot of code but also a much safer alternative. I’m also curious as to how it compares performance wise.

There’s a tool called jextract (linked below) that can generate foreign function bindings from header files, and we won’t have to do all the work we did above so the lots of code issue isn’t really an issue. I decided not to use jextract at all in this article because it’ll be like learning java multithreading without learning Java.

Maybe I’ll write on how to use jextract with something like SDL2 or LLVM? Or a machine learning library? It most likely won’t be as straightforward as simply calling jextract with some header files, but we’ll see 🙂

References

Source Code

Run with java 22+ : java --enable-preview --source 22 Runner.java