Skip to content

[Bug] Flaky auto-fetch #48

@mszczepanczyk

Description

@mszczepanczyk

Hi, the auto-fetch feature seems very flaky in docker. I've managed to fix a couple of issues (you can check the last commits in my fork: main...mszczepanczyk:domain-locker:main) but in the end I've given up. For some domains it works, for some it doesn't. Setting DNS_DUMPSTER_TOKEN doesn't help, haven't tried shodan.

Here's what happens when it does work:

Nov 20 11:12:24 homelab docker-domain-locker-start[2940573]: 🔵 [domain-info] Resolving domain info for: google.com
Nov 20 11:12:25 homelab docker-domain-locker-start[2940573]: 🟢 [whois] Got WHOIS data via whois-json for google.com
Nov 20 11:12:54 homelab docker-domain-locker-start[2940573]: 🟢 [domain-info] Successfully resolved: google.com

Here's what happens when it doesn't:

Nov 20 11:13:19 homelab docker-domain-locker-start[2940573]: 🔵 [domain-info] Resolving domain info for: mariusz.sh
Nov 20 11:13:19 homelab docker-domain-locker-start[2940573]: 🟢 [whois] Got WHOIS data via whois-json for mariusz.sh
Nov 20 11:13:29 homelab docker-domain-locker-start[2940573]: 🟡 [domain-info] SSL cert fetch failed: getaddrinfo ENOTFOUND mariusz.sh
Nov 20 11:13:48 homelab docker-domain-locker-start[2940573]: 🟢 [domain-info] Successfully resolved: mariusz.sh
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]: SyntaxError: Unexpected token '<', "<html>
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:  <h"... is not valid JSON
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:     at JSON.parse (<anonymous>)
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:     at parseJSONFromBytes (node:internal/deps/undici/undici:5329:19)
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:     at successSteps (node:internal/deps/undici/undici:5300:27)
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:     at fullyReadBody (node:internal/deps/undici/undici:1447:9)
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:     at async specConsumeBody (node:internal/deps/undici/undici:5309:7)
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:     at async fetchSubdomains (file:///app/dist/analog/server/chunks/routes/domain-subs.mjs:41:18)
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:     at async Object.handler (file:///app/dist/analog/server/chunks/routes/domain-subs.mjs:150:27)
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]: 🔴 [domain-subs] Error fetching from https://api.shodan.io/dns/domain/mariusz.sh: SyntaxError: Unexpected token '<', "<html>
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:  <h"... is not valid JSON
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]: 🔴 [domain-subs] DNSDumpster token is not configured
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]: 🔴 [domain-subs] Error fetching subdomains for mariusz.sh: Error: Both primary and fallback services failed

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions