feat: redesign Configuration class for v4

[B4nan]

Summary

• Replaces get()/set() with direct property access — config.headless instead of config.get('headless'). Both methods are removed (v4 breaking change).
• Constructor options now take highest priority — new Configuration({ headless: false }) works even when CRAWLEE_HEADLESS=true is set. New priority: constructor > env vars > crawlee.json > schema defaults.
• Immutable instances — assigning to a config property throws TypeError.
• Zod-based field definitions — schema, env var mapping, and defaults defined in one place (crawleeConfigFields).
• Extensible via subclassing — subclasses override protected static fields to register additional config fields (e.g. Apify SDK).

Closes #3080

🤖 Generated with Claude Code

[B4nan]

This should be ready for review now, one open question:

In v3, when an env var was set to an empty string (CRAWLEE_HEADLESS=''), it was treated as "set" and passed through type coercion:

• booleans: '' → false (empty string was in the falsy list alongside '0' and 'false')
• numbers: '' → 0 (just did +value, and +'' === 0)
• strings: '' → '' (passed through as-is)

The number behavior was almost certainly a bug — nobody sets CRAWLEE_PERSIST_STATE_INTERVAL_MILLIS='' expecting 0ms. The boolean case is more debatable — treating '' as falsy has some logic to it. The string case is also questionable (CRAWLEE_DEFAULT_DATASET_ID='' would break).

In the new Zod-based Configuration, readEnvVar() decides whether to pass the value to the schema or skip it. We have a few options:

Option A: '' = unset for everything (current state on this branch)
Empty strings are filtered out in readEnvVar, value falls through to schema defaults. Simple, consistent.
Slight behavioral change for booleans (CRAWLEE_HEADLESS='' now gives true instead of false), but arguably nobody does this intentionally.

Option B: '' = unset for numbers/strings, falsy for booleans
Preserves the one v3 behavior that made sense (boolean) while fixing the buggy ones (number/string).
Requires type-aware logic — either checking the schema type in readEnvVar, or using safeParse in resolveAll and falling through on failure.

Option C: '' = set for everything (full v3 compat)
Pass empty strings through to schema coercion unchanged. Booleans get false, numbers get 0, strings get ''. Preserves all v3 behavior including the bugs.

I'm leaning toward A — clean break, simple code, and nobody intentionally relies on this. Thoughts?

---

There is also a connected SDK PR: apify/apify-sdk-js#583

[janbuchar]

The number behavior was almost certainly a bug — nobody sets CRAWLEE_PERSIST_STATE_INTERVAL_MILLIS='' expecting 0ms.

Hard to disagree with that. My hunch would be that this should mean "unset", i.e., use the default.

The boolean case is more debatable — treating '' as falsy has some logic to it.

True, I always find myself wondering how to set an environment variable to boolean false. But I think that we can accept the fact that environment variables are strings.

The string case is also questionable (CRAWLEE_DEFAULT_DATASET_ID='' would break).

True, on the other hand, an empty string is a string, too. And if CRAWLE_SOMETHING='' means "use the default", we have no way to express an empty string. Not that I can think of an use case for that in Crawlee 🙂

Of the three proposed solutions, A is the elegant one, C is backwards compatible and B tries to bolt a type system onto environment variables, which might either be just the solution that we're looking for, or a road to hell of confusion and unforeseen edge cases.

Is there a known use case that would break with A? Do people run CRAWLEE_PURGE_ON_START= npm start, for instance?

[B4nan]

I think/hope people don't do this on purpose, I'd expect 99% of users will just go with 1/0. So a break here feels pretty fine.

[barjin]

Just a passing-by comment:

Option A: '' = unset for everything (current state on this branch)

This doesn't seem to be true currently; see the snippet below (the default value for both fields is true)

crawlee/packages/core/test/core/configuration.test.ts

Lines 109 to 115 in 4063bf6

	it('treats empty-string boolean env var as false', () => {
	setEnv('CRAWLEE_HEADLESS', '');
	expect(new Configuration().headless).toBe(false);
	setEnv('CRAWLEE_PURGE_ON_START', '');
	expect(new Configuration().purgeOnStart).toBe(false);
	});

Anyway, I'm also on board with Option A, it seems like the most logical / least surprising one.

[B4nan]

Yeah, the current implementation differs, I initially wanted to resolve the booleans mapping from '' to false, then realized we might as well break it and make it more sound.

[janbuchar]

A couple of nits. Let's test this live once they're resolved.

[janbuchar]

Missed opportunity to use an object shorthand - just rename config to configuration

[B4nan]

Yeah, when I was going through the diff and saw this, I was actually going to write here that I'd vote for renaming the option to just config 🙃

But then I realized we already have proxyConfiguration, so better keep the long form.

[janbuchar]

I would add "for example" to the list of services tracked by service locator so that it doesn't go stale later on.

[janbuchar]

Thanks for adding the explanation. I still think there might be a better place for this, I don't like modules to have side effects.

Perhaps you could make an issue so that we could resolve this later (way later)?

[B4nan]

#3615

[B4nan]

Almost forgot to resolve the priorities (#3484 (comment)), handled in the latest commit.