Swift Macro Client Testing

This is the fourth and final article in the Swift Macro Testing series. The purpose of the series is to explore a useful triad of testing approaches we use at Livefront when building Swift macros.

Over the course of this series, we’ve followed a test-driven approach to develop a Memoized Swift macro from scratch. The second article introduced expansion testing, and the third focused on unit testing. In this article, we’ll explore testing from a client perspective.

Client testing allows us to step outside our macro and experience it as users would. It’s a chance to “eat our own dog food” while it’s still in development. We know the macro intimately from the inside, but there’s more we can learn when we approach it from the outside.

Revisiting the MemoizedClient

In “ Swift Macro Expansion Testing ,” we created the Memoized macro using Xcode’s Swift Macro template. The template produced an executable target in our package called MemoizedClient, which we’ve largely ignored until now. It’s time to put it to work.

The target’s description in the package manifest is “A client of the library, which is able to use the macro in its own code.” This describes exactly what we want for client testing. However, testing executable targets can be finicky, so for simplicity we’ll convert it to a library. We’ll also add a dedicated client testing target. With these changes, the Package.swift manifest should look like this:

// swift-tools-version: 6.0

import PackageDescription
import CompilerPluginSupport

let package = Package(
    name: "Memoized",
    platforms: [.macOS(.v15)],
    products: [
        .library(name: "Memoized", targets: ["Memoized"]),
        .library(name: "MemoizedClient", targets: ["MemoizedClient"]),
    ],
    dependencies: [
        .package(url: "https://github.com/pointfreeco/swift-macro-testing", from: "0.6.0"),
        .package(url: "https://github.com/swiftlang/swift-syntax.git", from: "600.0.0-latest"),
    ],
    targets: [
        // Macro implementation that performs the source transformation of a macro.
        .macro(
            name: "MemoizedMacros",
            dependencies: [
                .product(name: "SwiftSyntaxMacros", package: "swift-syntax"),
                .product(name: "SwiftCompilerPlugin", package: "swift-syntax")
            ]
        ),

        // Library that exposes a macro as part of its API, which is used in client programs.
        .target(name: "Memoized", dependencies: ["MemoizedMacros"]),

        // A client of the library, which is able to use the macro in its own code.
        .target(name: "MemoizedClient", dependencies: ["Memoized"]),

        // A test target used to develop the macro implementation.
        .testTarget(
            name: "MemoizedTests",
            dependencies: [
                "MemoizedMacros",
                .product(name: "MacroTesting", package: "swift-macro-testing"),
                .product(name: "SwiftSyntaxMacrosTestSupport", package: "swift-syntax"),
            ]
        ),

        // A test target used to test the macro client.
        .testTarget(name: "MemoizedClientTests", dependencies: ["MemoizedClient"]),
    ]
)

We also need to make a few filesystem changes to accommodate the Swift Package Manager:

mv Sources/MemoizedClient/main.swift Sources/MemoizedClient/Samples.swift
mkdir -p Tests/MemoizedClientTests
touch Tests/MemoizedClientTests/SamplesTests.swift

A First Client Test

Continuing with our test-driven approach, let’s create a simple test in our new client testing file:

// Tests/MemoizedClientTests/SamplesTests.swift

import Testing

@testable import MemoizedClient

struct SamplesTests {
    @Test func fibonacciValues() {
        let sample = Samples()

        #expect(sample.fibonacci(1) == 1)
        #expect(sample.fibonacci(3) == 2)
        #expect(sample.fibonacci(5) == 5)
        #expect(sample.fibonacci(7) == 13)
    }
}

As expected, Xcode complains with a 🛑 ⁠Cannot find ‘Samples’ in scope error. Let’s add the code we’ve been envisioning since the start of the series to the Samples.swift file:

// Sources/MemoizedClient/Samples.swift

import Memoized

struct Samples {
    @Memoized
    func fibonacci(_ n: Int) -> Int {
        if n < 0 {
            0
        } else if n < 2 {
            n
        } else {
            memoizedFibonacci(n - 1) + memoizedFibonacci(n - 2)
        }
    }
}

Reasonable Restrictions

The Memoized macro has been thoroughly tested, so we’d expect our new test to pass. Surprisingly, it doesn’t. In fact, the macro expansion won’t even compile:

This is unexpected, given that we based the macro expansion on a working example from the first article, “ Exploring Memoization in Swift .” However, the error isn’t difficult to understand. Looking back at our working example, the memoized function was in a class, but our ⁠Samples is a struct. In Swift, functions that modify properties of a struct must be labeled as ⁠mutating.

There are several ways we could fix this:

1. Make both of the functions mutating
2. Change Sample from a struct to a class
3. Find a solution that doesn’t place unreasonable restrictions on the client

The first two options illustrate approaches to avoid. As a rule, macros should keep restrictions on the downstream client to an absolute minimum.

For the Memoized macro, it’s reasonable to restrict it to functions that return a value, since that’s a basic requirement of memoization. It’s also somewhat reasonable that the macro won’t work in an extension, since Swift doesn’t allow adding stored properties in extensions. While this is less than ideal, it’s only a restriction on placement.

Option one above is unreasonable not only because it’s conceptually wrong — labeling a pure function as ‘mutating’ — but because it would simultaneously prevent the macro from being used in classes. Option two demonstrates a clear overreach by placing restrictions on code outside the macro’s scope. Neither restriction is reasonable to impose on a client.

Non-Mutating Mutations

Fortunately, implementing the third option is straightforward. With a dictionary cache, we mutate a struct property with each key-value write. However, a reference-type cache like the one below would let us store keys and values without mutating the struct itself:

// Sources/Memoized/MemoizedCache.swift

import Foundation

public final class MemoizedCache {
    public var storage: [Int: Int]

    public init() {
        self.storage = [:]
    }
}

The ⁠MemoizedCache file needs to live in the Memoized target, since it’s the library that exposes the macro. By placing it there as a public class, it’s available when the macro expansions need it.

Now we need to update the ⁠MemoizedMacro.expansion function:

// Sources/MemoizedMacros/MemoizedMacro.swift

public struct MemoizedMacro: PeerMacro {
    public static func expansion(
        of node: AttributeSyntax,
        providingPeersOf declaration: some DeclSyntaxProtocol,
        in context: some MacroExpansionContext
    ) throws -> [DeclSyntax] {
        // ...
        let memoCacheDeclaration = "private var \(cacheVarName) = MemoizedCache()"
        // ...
        let memoFunctionDeclaration = """
            func \(memoFunctionName)(\(parameters)) -> \(returnType) {
                if let cached = \(cacheVarName).storage[n] {
                    return cached
                }
                let result = \(functionName)(\(invocationArguments))
                \(cacheVarName).storage[n] = result
                return result
            }
            """
        // ...
    }
}

This gets our client test compiling and passing. It also causes the ⁠memoizedFibonacciFunction expansion test to fail, which is expected since we’ve updated the expansion function. We should delete the test’s stale expansion closure and re-run tests to get the updated version.

Proof of Work

The client test demonstrates that ⁠Samples.fibonacci(_:) returns correct results. However, it doesn’t prove that memoization is working.

To gather that proof, let’s add a ⁠CountingSamples class where we can track how many times our function gets called. The class will contain a memoized ⁠fibonacci(_:) function and an un-memoized ⁠slowFibonacci(_:) function:

// Sources/MemoizedClient/CountingSamples.swift

import Memoized

class CountingSamples {
    private(set) var numberOfFunctionCalls = 0

    @Memoized
    func fibonacci(_ n: Int) -> Int {
        numberOfFunctionCalls += 1

        return if n < 0 {
            0
        } else if n < 2 {
            n
        } else {
            memoizedFibonacci(n - 1) + memoizedFibonacci(n - 2)
        }
    }

    func slowFibonacci(_ n: Int) -> Int {
        numberOfFunctionCalls += 1

        return if n < 0 {
            0
        } else if n < 2 {
            n
        } else {
            slowFibonacci(n - 1) + slowFibonacci(n - 2)
        }
    }
}

With a corresponding test file, we can compare the number of calls to each function. We even know what values to expect from the first article in our series:

// Tests/MemoizedClientTests/CountingSamplesTests.swift

import Testing

@testable import MemoizedClient

struct CountingSamplesTests {
    @Test func fibonacciFunctionCalls() {
        let sample = CountingSamples()

        #expect(sample.fibonacci(20) == 6_765)
        #expect(sample.numberOfFunctionCalls == 21)
    }

    @Test func slowFibonacciFunctionCalls() {
        let sample = CountingSamples()

        #expect(sample.slowFibonacci(20) == 6_765)
        #expect(sample.numberOfFunctionCalls == 21_891)
    }
}

The tests pass, providing proof that memoization is working as expected in the client code.

Signature Shake-Up

One thing that may have intrigued readers since “ Swift Macro Unit Testing ” is the comprehensive test suite for the ⁠invocationArguments(for:) function, which demonstrates that it can handle any combination of parameters.

So far, however, we’ve only applied Memoized to the ⁠fibonacci(_:) function, with its single unnamed parameter and integer return value. What happens when we apply it to a function with a very different signature?

// Sources/MemoizedClient/Samples.swift

struct Samples {
    @Memoized
    func nonsensicalAES(_ key: Int128 = -1, vector: String) -> String {
        // Imagine a very expensive operation
        "\(key)::\(vector)"
    }
}

When we add ⁠nonsensicalAES to ⁠Samples with the Memoized macro, we discover at least two problems in our implementation.

Time for More Tests

Since the ⁠nonsensicalAES function’s macro expansion isn’t compiling, our first step is to comment out the Memoized attribute. Next, let’s create a new expansion test using the function definition:

// Tests/MemoizedTests/MemoizedTests.swift

struct MemoizedTests {
    // ...
    @Test func memoizedNonsensicalAESFunction() {
        assertMacro {
            #"""
            @Memoized
            func nonsensicalAES(_ key: Int128 = -1, vector: String) -> String {
                // Imagine a very expensive operation
                "\(key)::\(vector)"
            }
            """#
        }
    }
}

Running tests generates the full macro expansion. Let’s copy the generated code and paste it verbatim into ⁠Samples.swift. Like the real macro expansion, this code doesn’t compile, but it gives us code to experiment with as we work toward a solution:

// Sources/MemoizedClient/Samples.swift

struct Samples {
    @Memoized
    func fibonacci(_ n: Int) -> Int {
        // ...
    }

    // @Memoized
    // func nonsensicalAES(_ key: Int128 = -1, vector: String) -> String {
    //     // Imagine a very expensive operation
    //     "\(key)::\(vector)"
    // }

    func nonsensicalAES(_ key: Int128 = -1, vector: String) -> String {
        // Imagine a very expensive operation
        "\(key)::\(vector)"
    }

    private var __macro_local_5CachefMu_ = MemoizedCache()

    func memoizedNonsensicalAES(_ key: Int128 = -1, vector: String) -> String {
        if let cached = __macro_local_5CachefMu_.storage[n] {
            return cached
        }
        let result = nonsensicalAES(key, vector: vector)
        __macro_local_5CachefMu_.storage[n] = result
        return result
    }
}

Correcting Past Assumptions

The ⁠memoizedNonsensicalAES function reveals two problems:

1. We hard-coded the ⁠fibonacci function’s ⁠n parameter as the key when accessing the ⁠MemoizedCache.storage[n] cache.
2. We assumed cached values would always be integers, but this function returns a string.

The results cache needs to be keyed by a unique representation of all parameter values. A simple solution might be:

let memoizedCacheKey = "\(key),\(vector)"

We’ll need to revisit MemoizedCache.swift to make two changes.

1. We need to change the storage dictionary’s key from an integer to a string.
2. We need to make ⁠MemoizedCache generic on the result type.

// Sources/Memoized/MemoizedCache.swift

public final class MemoizedCache<T> {
    public var storage: [String: T]

    public init() {
        self.storage = [:]
    }
}

Now let’s update our working code in ⁠Samples.swift:

func nonsensicalAES(_ key: Int128 = -1, vector: String) -> String {
    // Imagine a very expensive operation
    "\(key)::\(vector)"
}

private var __macro_local_5CachefMu_ = MemoizedCache<String>()

func memoizedNonsensicalAES(_ key: Int128 = -1, vector: String) -> String {
    let memoizedCacheKey = "\(key),\(vector)"
    if let cached = __macro_local_5CachefMu_.storage[memoizedCacheKey] {
        return cached
    }
    let result = nonsensicalAES(key, vector: vector)
    __macro_local_5CachefMu_.storage[memoizedCacheKey] = result
    return result
}

This resolves the compilation errors in our experimental code, suggesting we have a viable approach. Let’s copy this code back into our expansion test to record the specification we need to implement:

@Test("Memoized Nonsensical AES function")
func memoizedNonsensicalAESFunction() {
    assertMacro {
        #"""
        @Memoized
        func nonsensicalAES(_ key: Int128 = -1, vector: String) -> String {
            // Imagine a very expensive operation
            "\(key)::\(vector)"
        }
        """#
    } expansion: {
        #"""
        func nonsensicalAES(_ key: Int128 = -1, vector: String) -> String {
            // Imagine a very expensive operation
            "\(key)::\(vector)"
        }

        private var __macro_local_5CachefMu_ = MemoizedCache<String>()

        func memoizedNonsensicalAES(_ key: Int128 = -1, vector: String) -> String {
            let memoizedCacheKey = "\(key),\(vector)"
            if let cached = __macro_local_5CachefMu_.storage[memoizedCacheKey] {
                return cached
            }
            let result = nonsensicalAES(key, vector: vector)
            __macro_local_5CachefMu_.storage[memoizedCacheKey] = result
            return result
        }
        """#
    }
}

Putting It Back Together

Now we need to update the ⁠MemoizedMacro.expansion function to make everything compile again. The ⁠memoCacheDeclaration definition needs a generic argument for ⁠MemoizedCache<T>(), which we already have in ⁠returnType.

The ⁠memoFunctionDeclaration definition needs the ⁠memoizedCacheKey declaration and its replacement for ⁠n as the storage key. For now, let’s use ⁠\(n) as a placeholder:

// Sources/MemoizedMacros/MemoizedMacro.swift

public struct MemoizedMacro: PeerMacro {
    public static func expansion(
        of node: AttributeSyntax,
        providingPeersOf declaration: some DeclSyntaxProtocol,
        in context: some MacroExpansionContext
    ) throws -> [DeclSyntax] {
        // ...
        let returnType = functionReturnClause.type.trimmed.description

        let memoCacheDeclaration = "private var \(cacheVarName) = MemoizedCache<\(returnType)>()"
        // ...
        let memoFunctionDeclaration = """
            func \(memoFunctionName)(\(parameters)) -> \(returnType) {
                let memoizedCacheKey = "\\(n)"
                if let cached = \(cacheVarName).storage[memoizedCacheKey] {
                    return cached
                }
                let result = \(functionName)(\(invocationArguments))
                \(cacheVarName).storage[memoizedCacheKey] = result
                return result
            }
            """
        // ...
    }
}

When we run the tests, we’ll see failures in response to these changes. The ⁠memoizedFibonacciFunction test fails first, with differences that match our intended changes. We can delete its ⁠expansion closure and re-run tests to get the updated version.

The ⁠memoizedNonsensicalAESFunction test fails next, this time because of the ⁠memoizedCacheKey value:

Test "Memoized Nonsensical AES function" recorded an issue at MemoizedTests.swift:87:21: Issue recorded
Expanded output (+) differed from expected output (−). Difference: …
    // ...
    func memoizedNonsensicalAES(_ key: Int128 = -1, vector: String) -> String {
  −     let memoizedCacheKey = "\(key),\(vector)"
  +     let memoizedCacheKey = "\(n)"
        // ...
    }
Test "Memoized Nonsensical AES function" failed after 0.386 seconds with 1 issue.

Universal Key-Maker

As the test shows, we need a function that takes parameters and returns a unique key. Let’s specify how a ⁠keyExpression(for:) function should behave through unit tests:

// Tests/MemoizedTests/MemoizedMacroHelperTests.swift

extension MemoizedMacroHelperTests {
    @Test func keyExpressionZeroParams() throws {
        let result = MemoizedMacro.keyExpression(
            for: FunctionParameterListSyntax([])
        )
        #expect(result == "__no_args__")
    }

    @Test func keyExpressionOneNamedParameter() throws {
        let result = MemoizedMacro.keyExpression(
            for: FunctionParameterListSyntax(["arg1: String"])
        )
        #expect(result == #"\(arg1)"#)
    }

    @Test func keyExpressionOneUnnamedParameter() throws {
        let result = MemoizedMacro.keyExpression(
            for: FunctionParameterListSyntax(["_ arg1: String"])
        )
        #expect(result == #"\(arg1)"#)
    }

    @Test func keyExpressionTwoNamedParams() throws {
        let result = MemoizedMacro.keyExpression(
            for: FunctionParameterListSyntax(
                [
                    "arg1: String",
                    "arg2: Int"
                ]
            )
        )
        #expect(result == #"\(arg1),\(arg2)"#)
    }

    @Test func keyExpressionTwoUnnamedParams() throws {
        let result = MemoizedMacro.keyExpression(
            for: FunctionParameterListSyntax(
                [
                    "_ arg1: String",
                    "_ arg2: Int"
                ]
            )
        )
        #expect(result == #"\(arg1),\(arg2)"#)
    }

    @Test func keyExpressionMixOfNamedAndUnnamedParams() throws {
        let result = MemoizedMacro.keyExpression(
            for: FunctionParameterListSyntax(
                [
                    "_ arg1: String",
                    "arg2: Int"
                ]
            )
        )
        #expect(result == #"\(arg1),\(arg2)"#)
    }
}

The implementation of ⁠keyExpression(for:) is as follows:

// Sources/MemoizedMacros/MemoizedMacro.swift

extension MemoizedMacro {
    /// Creates a cache key expression based on the function's parameter names.
    ///
    /// - Parameter parameters: The list of function parameters from which to derive the key.
    ///
    /// - Returns: A string representing the key expression suitable for caching results.
    static func keyExpression(for parameters: FunctionParameterListSyntax) -> String {
        let parameterNames = parameters
            .map { "\\(\($0.secondName?.text ?? $0.firstName.text))" }
            .joined(separator: ",")

        return parameterNames.isEmpty ? "__no_args__" : parameterNames
    }
}

Now we need to integrate this into our expansion function:

// Sources/MemoizedMacros/MemoizedMacro.swift

public struct MemoizedMacro: PeerMacro {
    public static func expansion(
        of node: AttributeSyntax,
        providingPeersOf declaration: some DeclSyntaxProtocol,
        in context: some MacroExpansionContext
    ) throws -> [DeclSyntax] {
        // ...
        let keyExpression = keyExpression(for: parameters)

        let memoFunctionDeclaration = """
            func \(memoFunctionName)(\(parameters)) -> \(returnType) {
                let memoizedCacheKey = "\(keyExpression)"
                if let cached = \(cacheVarName).storage[memoizedCacheKey] {
                    return cached
                }
                let result = \(functionName)(\(invocationArguments))
                \(cacheVarName).storage[memoizedCacheKey] = result
                return result
            }
            """
        // ...
    }
}

Two More Client Tests

Now that everything is building and passing, let’s revisit the ⁠nonsensicalAES function that exposed flaws in our implementation. Let’s add some client tests to verify the macro is now working correctly:

// Tests/MemoizedClientTests/SamplesTests.swift

extension SamplesTests {
    @Test func nonsensicalAES() {
        let result = Samples().nonsensicalAES(42, vector: "qwerty")
        #expect(result == "42::qwerty")
    }

    @Test func nonsensicalAESDefaultKey() {
        let result = Samples().nonsensicalAES(vector: "asdfg")
        #expect(result == "-1::asdfg")
    }
}

Finally, let’s remove any experimental code and then uncomment the memoized function in the Samples file:

// Sources/MemoizedClient/Samples.swift

struct Samples {
    // ...

    @Memoized
    func nonsensicalAES(_ key: Int128 = -1, vector: String) -> String {
        // Imagine a very expensive operation
        "\(key)::\(vector)"
    }
}

After one final test run, all tests are passing.

All Together Now

Throughout this series, we began by exploring a basic optimization technique — memoization — and moved on to implementing it as a robust, reusable Swift macro. Along the way, we’ve explored three complementary testing approaches that together provide comprehensive coverage:

1. Expansion Testing: Our first approach allowed us to specify the macro’s overall behavior by declaring the expected output for a given input. This high-level testing gave us confidence that the macro would generate the correct code structure.
2. Unit Testing: As our implementation grew more complex, we moved to testing individual components in isolation. This approach enabled us to handle edge cases and ensure that each piece of our macro functioned correctly.
3. Client Testing: Finally, we stepped outside our implementation to experience the macro as users would. This revealed issues that weren’t apparent from the inside, leading to a more robust and flexible implementation.

Each testing approach provided unique insights that would have been difficult to discover with the others alone. Expansion testing gave us the big picture, unit testing ensured the details were correct, and client testing verified the practical usability.

The result is a ⁠@Memoized macro that transforms a computationally expensive function like this:

func fibonacci(_ n: Int) -> Int {
    if n < 0 {
        return 0
    } else if n < 2 {
        return n
    } else {
        return fibonacci(n - 1) + fibonacci(n - 2)
    }
}

Into an optimized version with just a single line of code:

@Memoized
func fibonacci(_ n: Int) -> Int {
    if n < 0 {
        return 0
    } else if n < 2 {
        return n
    } else {
        return memoizedFibonacci(n - 1) + memoizedFibonacci(n - 2)
    }
}

This implementation handles various parameter types, works in both classes and structs, and dramatically improves performance for computationally expensive functions.

Beyond the specific macro we’ve created, this series demonstrates a powerful approach to Swift macro development that you can apply to your own projects:

1. Start with a clear understanding of the problem you’re solving
2. Use expansion testing to outline the macro’s desired behavior
3. Develop component functionality with targeted unit tests
4. Verify real-world usability through client testing

By following this testing triad, you can create macros that are robust, flexible, and ready for production use. Whether you’re building tools for your team or developing libraries for the broader Swift community, this approach will help you create macros that developers will love to use.

We hope this series has inspired you to explore the possibilities of Swift macros in your own projects. Happy coding!

• Chris Sessions

  Software Engineer

  More from Chris