How does this work when the ModuleSource is used to instantiate a worker?

[bakkot]

The worker is a different thread. Is the idea that it would pause execution and call back into the main thread?

[Jamesernator]

This doesn't work with the importMetaHook as it can return things that may not be serializable across threads:

const mod = new ModuleSource(`console.log(import.meta.resolve("./foo.ts"))`, {
    importMetaHook(importMeta) {
         importMeta.resolve = (specifier) => { /* ... resolve here ... */ }
    },
});

I originally raised this same problem on the old compartments proposal, and the solution here could be the same, the TLDR would basically just be that we could have have the hooks be another module:

const mod = new ModuleSource(`console.log(import.meta.resolve("./foo.ts"))`, {
    // cloned as well to the thread, imported and used as hook for the module
    hooksModule: new ModuleSource(`
        export function importHook(moduleSource, specifier, ...etc) {

        }

        export function importMetaHook(moduleSource, importMeta) {
            
        }
    `),
});

Alternatively it may just be the case that we need to wait for shared structs:

shared struct SharedHook {
    importHook(specifier, ...etc) {
        
    }

    importMetaHook(importMeta) {
        importMeta.resolve = (specifier) => { /* ... */ }
    }
}

const mod = new ModuleSource(`console.log(import.meta.resolve("./foo.ts"))`, new SharedHook());

[legendecas]

The ModuleSource should be transferable, and a JavaScript callback importHook makes it untransferable. The source phase representation should be stateless, and can be efficiently shared with workers, and instantiated multiple times.

Taking WebAssembly.Module as a comparison, WebAssembly.Module is stateless, and once instantiated with imports, a WebAssembly.Instance is created.

In the current Stage 3 source phase import proposal, the class relations are:

• source phase (stateless): ModuleSource, WebAssembly.Module,
• instance phase (stateful): ModuleInstance(not a proposal yet?), WebAssembly.Instance.

[Jamesernator]

The ModuleSource should be transferable, and a JavaScript callback importHook makes it untransferable.

This is an unfortunate divergence from the previous compartments proposal that had split ModuleSource and Module, the former was clonable but the latter not, whereas this proposal makes ModuleSource conditionally clonable.

The source phase representation should be stateless, and can be efficiently shared with workers, and instantiated multiple times.

My suggestion(s) would not change this, for the hooksModule idea it would simply change ModuleSourcesEqual to also check if the hooks are equal, for the shared struct idea it would simply check the associated hook struct was identical.

Like the way import.meta.url works when host-created module sources are cloned is basically the same as if a module source carried along with it a host-defined hooksModule/shared-handler.

i.e. Essentially it would as if all (browser) host sources were created like (assuming the idea with shared structs, the hooksModule idea would still be generally similar):

shared struct BuiltinImportHook {
    #url;

    constructor(url) {
        this.#url = url;
    }

    async importHook(specifier, ...etc) {
        const resolvedUrl = new URL(specifier, this.#url);
        // ... fetch etc
        return cache.getOrInsertComputed(
            resolvedUrl, 
            () => new ModuleSource(sourceText, new BuiltinImportHook(resolvedUrl)),
        );
    }
}


import source fooSource from "./foo.js";
// equivalent to
const fooSource = new ModuleSource(
    readTextSomehow("./foo.js"), 
    new BuiltinImportHook(import.meta.resolve("./foo.js"),
);

// clones the source, keeps the same handler regardless of host-defined or user-defined 
const clonedSource = structuredClone(fooSource);