Skip to content

Redesign repo structure and get domains list from multiple sources #44

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
piquark6046 wants to merge 8 commits into
base: main
Choose a base branch
from
Draft

Redesign repo structure and get domains list from multiple sources #44

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
3eb385d
refactor: design monorepos
piquark6046 Jan 14, 2026
1b9724c
ci: fix ESLint chain
piquark6046 Jan 14, 2026
2973ddc
ci: expand ESLint checking to all Typescript files
piquark6046 Jan 14, 2026
f2bf1af
refactor: split IAB sellers downloader
piquark6046 Jan 15, 2026
0173bdb
refactor: update pacakge.json in `builder`
piquark6046 Jan 15, 2026
b7c2fd8
fix: `baseUrl` in `tsconfig.json` leads fatal of `tsx` execution
piquark6046 Jan 15, 2026
45468db
chore: add http-server utils
piquark6046 Jan 16, 2026
e4dcb6a
chore: update .gitignore and package.json for new monorepo structure
piquark6046 Jan 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
node_modules
dist
sources/src/#generated-*.ts
sources/src/#generated-*.ts
.buildcache
32 changes: 32 additions & 0 deletions builder/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
{
"name": "@filteringdev/tinyshield-builder",
"private": true,
"type": "module",
"scripts": {
"lint": "tsc --noEmit && eslint **/*.ts",
"build": "tsx source/build.ts",
"debug": "tsx source/debug.ts",
"cache": "tsx source/cache.ts",
"clean": "rm -rf dist && rm -rf .buildcache"
},
"dependencies": {
"@types/node": "^24.10.8"
},
"devDependencies": {
"@adguard/agtree": "^3.4.3",
"@npmcli/package-json": "^7.0.4",
"@types/npmcli__package-json": "^4.0.4",
"@types/semver": "^7.7.1",
"@typescriptprime/parsing": "^1.0.4",
"@typescriptprime/securereq": "^1.0.1",
"chokidar": "^5.0.0",
"esbuild": "^0.27.2",
"eslint": "^9.39.2",
"semver": "^7.7.3",
"tldts": "^7.0.19",
"tsx": "^4.21.0",
"typescript": "^5.9.3",
"typescript-eslint": "^8.53.0",
"zod": "^4.3.5"
}
}
0 sources/banner.txt → builder/source/banner.txt
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion builder.ts → builder/source/builder.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ let AttachHeaderPath = `/tmp/${crypto.randomUUID()}`
Fs.writeFileSync(AttachHeaderPath, ConvertedHeader, { encoding: 'utf-8', mode: 0o700 })
console.log('Written temporary header file to:', AttachHeaderPath)
await ESBuild.build({
entryPoints: ['./sources/src/index.ts'],
entryPoints: ['./sources/index.ts'],
bundle: true,
minify: BuildType === 'production',
define: {
Expand Down
38 changes: 38 additions & 0 deletions builder/source/references/iabsellers.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
import * as Zod from 'zod'
import { HTTPSRequest } from '@typescriptprime/securereq'


const IABSellersJsonURL = 'https://info.ad-shield.io/sellers.json'

export async function FetchIABSellersJsonData(): Promise<string[]> {
const IABSellersJsonResponse: { StatusCode: number, Headers: Record<string, string | string[]>, Body: unknown } = await HTTPSRequest(new URL(IABSellersJsonURL), { ExpectedAs: 'JSON' })
let IABSellersJsonData =IABSellersJsonResponse.Body as {
// eslint-disable-next-line @typescript-eslint/naming-convention
sellers: Array<{
// eslint-disable-next-line @typescript-eslint/naming-convention
seller_id: number,
// eslint-disable-next-line @typescript-eslint/naming-convention
seller_type: string,
// eslint-disable-next-line @typescript-eslint/naming-convention
name: string,
// eslint-disable-next-line @typescript-eslint/naming-convention
domain: string
}>
}
IABSellersJsonData = await Zod.object({
sellers: Zod.array(Zod.object({
seller_id: Zod.number(),
seller_type: Zod.string(),
name: Zod.string(),
domain: Zod.string().refine(D => {
try {
new URL(`https://${D}`)
} catch {
return false
}
return true
})
}))
}).parseAsync(IABSellersJsonData)
return [...new Set(IABSellersJsonData.sellers.map(S => S.domain))]
}
33 changes: 33 additions & 0 deletions builder/source/utils/http-server.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
import * as HTTP from 'node:http'
import * as Fs from 'node:fs'

function IsLoopBack(IP: string) {
return IP === '127.0.0.1' || IP === '::1' || IP === '::ffff:127.0.0.1'
}

export function RunDebugServer(Port: number, FileName: string[], ShouldPreventHTTPResponse: boolean) {
const HTTPServer = HTTP.createServer((Req, Res) => {
if (!FileName.includes(Req.url?.substring(1) || '')) {
Res.writeHead(404)
Res.end()
return
} else if (!IsLoopBack(Req.socket.remoteAddress)) {
Res.writeHead(403)
Res.end()
return
} else if (ShouldPreventHTTPResponse || !Fs.existsSync(`${process.cwd()}/dist/${Req.url?.substring(1)}`)) {

Check failure

Code scanning / CodeQL

Uncontrolled data used in path expression High

This path depends on a
user-provided value
Loading
.

Copilot Autofix

AI about 7 hours ago

In general, to fix uncontrolled path usage, normalize the user input, resolve it relative to a known safe root directory, and then verify that the resulting path is contained within that root. If the normalized path escapes the root, reject the request. For this server, the safe root is effectively ${process.cwd()}/dist. We should use path.resolve to join this root with the requested path, then ensure the resolved path starts with the root directory. This prevents directory traversal and absolute path attacks.

The best fix here is: (1) introduce an import of Node’s path module; (2) compute const distRoot = path.join(process.cwd(), 'dist') once per request (or outside the handler if appropriate); (3) take Req.url?.substring(1) || '' as the requested path segment, resolve it with path.resolve(distRoot, requestedPath), and (4) check that the resolved path starts with distRoot + path.sep or equals distRoot. If not, return 403. Then use this sanitized resolvedPath for both Fs.existsSync and Fs.readFileSync. This preserves existing functionality (files are still only served if in FileName, from loopback, and if they exist) while adding robust path validation.

Concretely in builder/source/utils/http-server.ts:

  • Add import * as Path from 'node:path' at the top.
  • Inside the request handler, compute const distRoot = Path.join(process.cwd(), 'dist') and const requestPath = Req.url?.substring(1) || ''.
  • Derive const resolvedPath = Path.resolve(distRoot, requestPath).
  • Before the FileName.includes/loopback/exists checks that depend on the path, add a guard: if !resolvedPath.startsWith(distRoot + Path.sep) && resolvedPath !== distRoot, respond with 403 and return.
  • Replace both occurrences of `${process.cwd()}/dist/${Req.url?.substring(1)}` with resolvedPath.

No additional helper methods are strictly necessary beyond the new Path import and local variables.

Suggested changeset 1
builder/source/utils/http-server.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/builder/source/utils/http-server.ts b/builder/source/utils/http-server.ts
--- a/builder/source/utils/http-server.ts
+++ b/builder/source/utils/http-server.ts
@@ -1,5 +1,6 @@
 import * as HTTP from 'node:http'
 import * as Fs from 'node:fs'
+import * as Path from 'node:path'
 
 function IsLoopBack(IP: string) {
   return IP === '127.0.0.1' || IP === '::1' || IP === '::ffff:127.0.0.1'
@@ -7,7 +8,18 @@
 
 export function RunDebugServer(Port: number, FileName: string[], ShouldPreventHTTPResponse: boolean) {
   const HTTPServer = HTTP.createServer((Req, Res) => {
-    if (!FileName.includes(Req.url?.substring(1) || '')) {
+    const distRoot = Path.join(process.cwd(), 'dist')
+    const requestPath = Req.url?.substring(1) || ''
+    const resolvedPath = Path.resolve(distRoot, requestPath)
+
+    // Ensure the resolved path stays within the dist root to prevent directory traversal
+    if (!resolvedPath.startsWith(distRoot + Path.sep) && resolvedPath !== distRoot) {
+      Res.writeHead(403)
+      Res.end()
+      return
+    }
+
+    if (!FileName.includes(requestPath)) {
       Res.writeHead(404)
       Res.end()
       return
@@ -15,13 +27,13 @@
       Res.writeHead(403)
       Res.end()
       return
-    } else if (ShouldPreventHTTPResponse || !Fs.existsSync(`${process.cwd()}/dist/${Req.url?.substring(1)}`)) {
+    } else if (ShouldPreventHTTPResponse || !Fs.existsSync(resolvedPath)) {
       Res.writeHead(503)
       Res.end('File not built yet.')
       return
     }
 
-    const Content = Fs.readFileSync(`${process.cwd()}/dist/${Req.url?.substring(1)}`, 'utf-8')
+    const Content = Fs.readFileSync(resolvedPath, 'utf-8')
     Res.writeHead(200, {
       'content-type': 'application/javascript; charset=utf-8',
       'content-length': new TextEncoder().encode(Content).byteLength.toString()
EOF
@@ -1,5 +1,6 @@
import * as HTTP from 'node:http'
import * as Fs from 'node:fs'
import * as Path from 'node:path'

function IsLoopBack(IP: string) {
return IP === '127.0.0.1' || IP === '::1' || IP === '::ffff:127.0.0.1'
@@ -7,7 +8,18 @@

export function RunDebugServer(Port: number, FileName: string[], ShouldPreventHTTPResponse: boolean) {
const HTTPServer = HTTP.createServer((Req, Res) => {
if (!FileName.includes(Req.url?.substring(1) || '')) {
const distRoot = Path.join(process.cwd(), 'dist')
const requestPath = Req.url?.substring(1) || ''
const resolvedPath = Path.resolve(distRoot, requestPath)

// Ensure the resolved path stays within the dist root to prevent directory traversal
if (!resolvedPath.startsWith(distRoot + Path.sep) && resolvedPath !== distRoot) {
Res.writeHead(403)
Res.end()
return
}

if (!FileName.includes(requestPath)) {
Res.writeHead(404)
Res.end()
return
@@ -15,13 +27,13 @@
Res.writeHead(403)
Res.end()
return
} else if (ShouldPreventHTTPResponse || !Fs.existsSync(`${process.cwd()}/dist/${Req.url?.substring(1)}`)) {
} else if (ShouldPreventHTTPResponse || !Fs.existsSync(resolvedPath)) {
Res.writeHead(503)
Res.end('File not built yet.')
return
}

const Content = Fs.readFileSync(`${process.cwd()}/dist/${Req.url?.substring(1)}`, 'utf-8')
const Content = Fs.readFileSync(resolvedPath, 'utf-8')
Res.writeHead(200, {
'content-type': 'application/javascript; charset=utf-8',
'content-length': new TextEncoder().encode(Content).byteLength.toString()
Copilot is powered by AI and may make mistakes. Always verify output.
Res.writeHead(503)
Res.end('File not built yet.')
return
}

const Content = Fs.readFileSync(`${process.cwd()}/dist/${Req.url?.substring(1)}`, 'utf-8')

Check failure

Code scanning / CodeQL

Uncontrolled data used in path expression High

This path depends on a
user-provided value
Loading
.

Copilot Autofix

AI about 7 hours ago

In general, the fix is to ensure that any path derived from user input is constrained to a safe root directory. That means: (1) normalize the combined path (root + user input) using path.resolve (and optionally fs.realpathSync), and (2) verify that the normalized path is still under the intended root (dist), rejecting the request otherwise. Additionally, avoid using the raw URL for filesystem access; use the validated/normalized path instead.

For this snippet, the best minimal fix without changing functionality is:

  • Introduce a constant DIST_ROOT that holds the absolute path to the dist directory, computed via path.resolve(process.cwd(), 'dist').
  • Import Node’s path module as import * as Path from 'node:path'.
  • For each usage of `${process.cwd()}/dist/${Req.url?.substring(1)}`, compute a safe path:
    • Take the requested path segment Req.url?.substring(1) || ''.
    • Normalize it with Path.normalize.
    • Resolve it against DIST_ROOT with Path.resolve(DIST_ROOT, requestedPath).
    • Optionally (more robustly) pass that resolved path through Fs.realpathSync in a try/catch to handle missing files cleanly.
    • Check that the final resolved path starts with DIST_ROOT + Path.sep (or is exactly DIST_ROOT) to ensure it doesn’t escape the root. If it does, respond with 403 and return.
  • Replace both Fs.existsSync(...) and Fs.readFileSync(...) to use this validated resolved path (safePath) instead of reconstructing the string each time.

Concretely, inside RunDebugServer:

  • Define const DIST_ROOT = Path.resolve(process.cwd(), 'dist') near the top of the function.
  • Inside the request handler, once you’ve passed the FileName.includes(...) and loopback checks, compute the safe path as described.
  • Use that safe path for existsSync and readFileSync.
  • If resolving or realpathSync throws or if the path check fails, respond with 403 (for traversal) or 503/404 as appropriate and return.

This keeps the existing behavior (only serving files in FileName, still returning 503 when the file is not built) while adding proper path safety.

Suggested changeset 1
builder/source/utils/http-server.ts

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/builder/source/utils/http-server.ts b/builder/source/utils/http-server.ts
--- a/builder/source/utils/http-server.ts
+++ b/builder/source/utils/http-server.ts
@@ -1,13 +1,18 @@
 import * as HTTP from 'node:http'
 import * as Fs from 'node:fs'
+import * as Path from 'node:path'
 
 function IsLoopBack(IP: string) {
   return IP === '127.0.0.1' || IP === '::1' || IP === '::ffff:127.0.0.1'
 }
 
 export function RunDebugServer(Port: number, FileName: string[], ShouldPreventHTTPResponse: boolean) {
+  const DIST_ROOT = Path.resolve(process.cwd(), 'dist')
+
   const HTTPServer = HTTP.createServer((Req, Res) => {
-    if (!FileName.includes(Req.url?.substring(1) || '')) {
+    const rawPath = Req.url?.substring(1) || ''
+
+    if (!FileName.includes(rawPath)) {
       Res.writeHead(404)
       Res.end()
       return
@@ -15,13 +13,33 @@
       Res.writeHead(403)
       Res.end()
       return
-    } else if (ShouldPreventHTTPResponse || !Fs.existsSync(`${process.cwd()}/dist/${Req.url?.substring(1)}`)) {
+    }
+
+    const normalizedRequestedPath = Path.normalize(rawPath)
+    const resolvedPath = Path.resolve(DIST_ROOT, normalizedRequestedPath)
+
+    if (resolvedPath !== DIST_ROOT && !resolvedPath.startsWith(DIST_ROOT + Path.sep)) {
+      Res.writeHead(403)
+      Res.end()
+      return
+    }
+
+    let safePath: string
+    try {
+      safePath = Fs.realpathSync(resolvedPath)
+    } catch {
       Res.writeHead(503)
       Res.end('File not built yet.')
       return
     }
 
-    const Content = Fs.readFileSync(`${process.cwd()}/dist/${Req.url?.substring(1)}`, 'utf-8')
+    if (ShouldPreventHTTPResponse || !Fs.existsSync(safePath)) {
+      Res.writeHead(503)
+      Res.end('File not built yet.')
+      return
+    }
+
+    const Content = Fs.readFileSync(safePath, 'utf-8')
     Res.writeHead(200, {
       'content-type': 'application/javascript; charset=utf-8',
       'content-length': new TextEncoder().encode(Content).byteLength.toString()
EOF
@@ -1,13 +1,18 @@
import * as HTTP from 'node:http'
import * as Fs from 'node:fs'
import * as Path from 'node:path'

function IsLoopBack(IP: string) {
return IP === '127.0.0.1' || IP === '::1' || IP === '::ffff:127.0.0.1'
}

export function RunDebugServer(Port: number, FileName: string[], ShouldPreventHTTPResponse: boolean) {
const DIST_ROOT = Path.resolve(process.cwd(), 'dist')

const HTTPServer = HTTP.createServer((Req, Res) => {
if (!FileName.includes(Req.url?.substring(1) || '')) {
const rawPath = Req.url?.substring(1) || ''

if (!FileName.includes(rawPath)) {
Res.writeHead(404)
Res.end()
return
@@ -15,13 +13,33 @@
Res.writeHead(403)
Res.end()
return
} else if (ShouldPreventHTTPResponse || !Fs.existsSync(`${process.cwd()}/dist/${Req.url?.substring(1)}`)) {
}

const normalizedRequestedPath = Path.normalize(rawPath)
const resolvedPath = Path.resolve(DIST_ROOT, normalizedRequestedPath)

if (resolvedPath !== DIST_ROOT && !resolvedPath.startsWith(DIST_ROOT + Path.sep)) {
Res.writeHead(403)
Res.end()
return
}

let safePath: string
try {
safePath = Fs.realpathSync(resolvedPath)
} catch {
Res.writeHead(503)
Res.end('File not built yet.')
return
}

const Content = Fs.readFileSync(`${process.cwd()}/dist/${Req.url?.substring(1)}`, 'utf-8')
if (ShouldPreventHTTPResponse || !Fs.existsSync(safePath)) {
Res.writeHead(503)
Res.end('File not built yet.')
return
}

const Content = Fs.readFileSync(safePath, 'utf-8')
Res.writeHead(200, {
'content-type': 'application/javascript; charset=utf-8',
'content-length': new TextEncoder().encode(Content).byteLength.toString()
Copilot is powered by AI and may make mistakes. Always verify output.
Res.writeHead(200, {
'content-type': 'application/javascript; charset=utf-8',
'content-length': new TextEncoder().encode(Content).byteLength.toString()
})
Res.end(Content)
})

HTTPServer.listen(Port)
}
7 changes: 7 additions & 0 deletions builder/tsconfig.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"extends": "../tsconfig.json",
"include": [
"source/**/*.ts",
"test/**/*.ts"
]
}
23 changes: 6 additions & 17 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
"build:userscript": "tsx builder.ts -- production",
"build": "npm run build:interface && npm run build:userscript",
"debug": "tsx builder.ts -- development",
"lint": "tsc --noEmit && eslint sources/**/*.ts"
"lint": "npm run lint -w builder && npm run lint -w userscript"
},
"keywords": [
"Ad-Shield"
Expand All @@ -28,20 +28,9 @@
"url": "git+https://github.com/FilteringDev/tinyShield.git"
},
"license": "MPL-2.0",
"dependencies": {
"@types/node": "^24.9.2"
},
"devDependencies": {
"@npmcli/package-json": "^7.0.4",
"@types/npmcli__package-json": "^4.0.4",
"@types/semver": "^7.7.1",
"esbuild": "^0.27.0",
"eslint": "^9.38.0",
"semver": "^7.7.3",
"tsx": "^4.21.0",
"typescript": "^5.9.3",
"typescript-eslint": "^8.46.2",
"zod": "^4.3.5"
},
"packageManager": "npm@11.5.1+"
"packageManager": "npm@11.5.1+",
"workspaces": [
"userscript",
"builder"
]
}
9 changes: 0 additions & 9 deletions sources/esbuild.inject.ts
View file
Open in desktop

This file was deleted.

8 changes: 7 additions & 1 deletion tsconfig.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@
"moduleResolution": "NodeNext",
"removeComments": false,
"alwaysStrict": false,
"skipLibCheck": true
"skipLibCheck": true,
"paths": {
"@builder/*": ["./builder/source/*"],
"@userscript/*": ["./userscript/source/*"],
"@root/*": ["./source/*"],
"@reporoot/*": ["./*"]
}
}
}
11 changes: 11 additions & 0 deletions userscript/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"name": "@filteringdev/tinyshield-userscript",
"private": true,
"type": "module",
"scripts": {
"lint": "tsc --noEmit && eslint **/*.ts"
},
"devDependencies": {
"@types/web": "^0.0.317"
}
}
0 sources/src/as-weakmap.ts → userscript/source/as-weakmap.ts
View file
Open in desktop
File renamed without changes.
0 sources/src/index.ts → userscript/source/index.ts
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion sources/interface.ts → userscript/source/interface.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@
* - See Git history at https://github.com/FilteringDev/tinyShield for detailed authorship information.
*/

export { RunTinyShieldUserscript } from './src/index.js'
export { RunTinyShieldUserscript } from './index.js'
0 sources/src/utils.ts → userscript/source/utils.ts
View file
Open in desktop
File renamed without changes.
6 changes: 6 additions & 0 deletions userscript/tsconfig.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"extends": "../tsconfig.json",
"include": [
"source/**/*.ts"
]
}