chore: add resource-import skill + book import scripts + server docs update
This commit is contained in:
Jens Reinemann
parent
5edaa06ecf
commit
c16c9fff97
7 changed files with 1057 additions and 44 deletions
181
.github/skills/resource-import/SKILL.md
vendored
Normal file
181
.github/skills/resource-import/SKILL.md
vendored
Normal file
|
|
@ -0,0 +1,181 @@
|
|||
---
|
||||
name: resource-import
|
||||
description: "Freie Bücher und andere Ressourcen (ePub, PDF) mit vollständigen Metadaten recherchieren und in die Bollwerk-App importieren. Nutze diesen Skill wenn neue Ressourcen gesucht, zusammengestellt oder importiert werden sollen. Trigger-Phrasen: 'Bücher importieren', 'freie Bücher', 'Ressourcen importieren', 'neue Bücher', 'Gutenberg', 'ePub', 'Ressource hinzufügen', 'Import Ressourcen', 'Bibliothek erweitern', '10 Bücher', 'Leseliste'."
|
||||
---
|
||||
|
||||
# Skill: Ressourcen-Import (freie Bücher mit Metadaten)
|
||||
|
||||
Recherchiert freie Bücher (Public Domain / Creative Commons) und importiert sie als Ressourcen in die Bollwerk-App. Enthält den gesamten Workflow: Auswahl → Metadaten → Download → Upload.
|
||||
|
||||
---
|
||||
|
||||
## Kontext
|
||||
|
||||
Die Bollwerk-App hat eine **Ressourcen-Bibliothek** mit ePub/PDF-Dateien. Der Server speichert pro Ressource:
|
||||
|
||||
| Feld | Typ | Beschreibung |
|
||||
| ------------ | ----------- | --------------------------------------------- |
|
||||
| guid | String | UUID, wird beim Upload generiert |
|
||||
| title | String | Buchtitel |
|
||||
| author | String? | Autor(en) |
|
||||
| description | String | 2–3 Sätze, deutsch, themenrelevant |
|
||||
| tags | List<String>| Kategorie-Tags (siehe unten) |
|
||||
| fileFormat | String | `epub` oder `pdf` |
|
||||
| mimeType | String | `application/epub+zip` / `application/pdf` |
|
||||
| fileSize | Long | Wird beim Upload automatisch gesetzt |
|
||||
| releaseDate | String? | Erstveröffentlichung (ISO oder Jahreszahl) |
|
||||
| language | String? | ISO-639-1 Sprachcode (`en`, `de`, etc.) |
|
||||
| edition | String? | Ausgabe/Übersetzung |
|
||||
| createdAt | Long | Epoch-Millisekunden (automatisch) |
|
||||
| updatedAt | Long | Epoch-Millisekunden (automatisch) |
|
||||
| downloadUrl | String | Server setzt dies auf `/api/resources/{guid}/download` |
|
||||
|
||||
### Bekannte Tags
|
||||
|
||||
- `nachschlagewerk` – Sachbuch, Lexikon, Ratgeber
|
||||
- `handbuch` – Praxisorientiertes How-To
|
||||
- `novel` – Belletristik, Roman, Novelle
|
||||
- `abenteuer` – Abenteuer-/Entdeckerthematik
|
||||
- `quellenbuch` – Inspirationsquelle / Settingbuch
|
||||
|
||||
Neue Tags dürfen bei Bedarf ergänzt werden, sollten aber zur bestehenden Systematik passen (Singular, Kleinbuchstaben, deutsch).
|
||||
|
||||
---
|
||||
|
||||
## Quellen für freie Bücher
|
||||
|
||||
| Quelle | URL | Formate | Lizenz |
|
||||
| -------------------------- | ----------------------------------- | ------------ | ------------ |
|
||||
| Project Gutenberg | https://www.gutenberg.org | ePub, txt | Public Domain |
|
||||
| Standard Ebooks | https://standardebooks.org | ePub | Public Domain |
|
||||
| Internet Archive (OpenLib) | https://archive.org | ePub, PDF | Gemischt |
|
||||
| ManyBooks | https://manybooks.net | ePub, PDF | Public Domain |
|
||||
|
||||
### Gutenberg-Download-URLs (Priorität)
|
||||
|
||||
```
|
||||
1. https://www.gutenberg.org/ebooks/{id}.epub.noimages (klein, schnell)
|
||||
2. https://www.gutenberg.org/ebooks/{id}.epub3.images (mit Bildern)
|
||||
3. https://www.gutenberg.org/cache/epub/{id}/pg{id}.epub (Fallback)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Auswahl (Recherche)
|
||||
|
||||
Wenn der User z.B. sagt "importiere 10 gute freie Bücher":
|
||||
|
||||
1. **Thema klären** – Was für Bücher? (Klassiker, Survival, Philosophie, Horror, etc.)
|
||||
2. **Lizenz sicherstellen** – Nur Public Domain oder explizit freie Lizenzen.
|
||||
3. **Duplikate prüfen** – Über `GET /api/resources` oder lokal prüfen, was bereits vorhanden ist.
|
||||
4. **Liste zusammenstellen** – Pro Buch:
|
||||
- Gutenberg-ID (oder Download-URL)
|
||||
- Titel, Autor
|
||||
- Deutsche Beschreibung (2–3 Sätze, für den Bollwerk-Kontext relevant)
|
||||
- Tags, Sprache, Erscheinungsjahr, Edition/Übersetzung
|
||||
|
||||
### 2. Import-Skript generieren/aktualisieren
|
||||
|
||||
Das Projekt hat bereits `import-books.py` als Referenzimplementierung. Für neue Imports:
|
||||
|
||||
- Entweder das bestehende Skript mit neuer BOOKS-Liste anpassen
|
||||
- Oder ein neues spezialisiertes Skript erstellen
|
||||
|
||||
**Skript-Struktur** (Python, nur stdlib):
|
||||
|
||||
```python
|
||||
BOOKS = [
|
||||
{
|
||||
"gutenberg_id": 132,
|
||||
"title": "The Art of War",
|
||||
"author": "Sun Tzu",
|
||||
"description": "Deutsche Beschreibung, 2-3 Sätze.",
|
||||
"tags": ["nachschlagewerk", "handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "-500",
|
||||
"edition": "Lionel Giles Translation, 1910",
|
||||
},
|
||||
# ...
|
||||
]
|
||||
```
|
||||
|
||||
### 3. Ausführung
|
||||
|
||||
```powershell
|
||||
# Voraussetzungen
|
||||
$env:BOLLWERK_ADMIN_USER = "<admin-username>"
|
||||
$env:BOLLWERK_ADMIN_PASS = "<admin-password>"
|
||||
|
||||
# Import starten
|
||||
python import-books.py
|
||||
|
||||
# Nur Download testen (kein Upload)
|
||||
python import-books.py --dry-run
|
||||
```
|
||||
|
||||
### 4. Verifizierung
|
||||
|
||||
Nach dem Import:
|
||||
|
||||
1. Server-Health prüfen: `GET /api/health`
|
||||
2. Ressourcen-Liste abrufen: `GET /api/resources` (Auth-Token nötig)
|
||||
3. Stichprobe: Ein Buch per `GET /api/resources/{guid}/download` herunterladen
|
||||
|
||||
---
|
||||
|
||||
## API-Endpunkte (Admin)
|
||||
|
||||
| Methode | Endpunkt | Beschreibung |
|
||||
| ------- | ------------------------------ | ------------------------------- |
|
||||
| POST | `/api/admin/resources` | Ressource hochladen (multipart) |
|
||||
| PUT | `/api/admin/resources/{guid}` | Metadaten aktualisieren |
|
||||
| DELETE | `/api/admin/resources/{guid}` | Ressource löschen |
|
||||
| GET | `/api/admin/resources/tags` | Alle Tags mit Häufigkeit |
|
||||
| POST | `/api/admin/resources/analyze` | Datei analysieren (ohne Speichern) |
|
||||
|
||||
### Multipart-Upload-Format
|
||||
|
||||
```
|
||||
POST /api/admin/resources
|
||||
Authorization: Bearer <token>
|
||||
Content-Type: multipart/form-data; boundary=...
|
||||
|
||||
Part 1: name="metadata", Content-Type: application/json
|
||||
→ ResourceDto als JSON
|
||||
|
||||
Part 2: name="file", filename="<guid>.epub"
|
||||
→ Datei-Bytes (max 25 MB)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Qualitätskriterien für Beschreibungen
|
||||
|
||||
- **Sprache**: Deutsch
|
||||
- **Länge**: 2–3 Sätze
|
||||
- **Inhalt**: Was ist das Buch? Warum ist es relevant/interessant?
|
||||
- **Keine Spoiler** bei Belletristik
|
||||
- **Kontext-Bezug**: Wenn möglich, Relevanz für Rollenspiel/Abenteuer/Strategie/Überleben hervorheben (passend zum Bollwerk-Thema)
|
||||
|
||||
---
|
||||
|
||||
## Checkliste vor Import
|
||||
|
||||
- [ ] Alle Bücher sind Public Domain oder frei lizenziert
|
||||
- [ ] Keine Duplikate mit bestehenden Ressourcen
|
||||
- [ ] Deutsche Beschreibungen vorhanden
|
||||
- [ ] Tags passen zur bestehenden Systematik
|
||||
- [ ] Gutenberg-IDs / Download-URLs verifiziert
|
||||
- [ ] `--dry-run` erfolgreich (alle Downloads funktionieren)
|
||||
- [ ] Env-Vars für Admin-Credentials gesetzt
|
||||
|
||||
---
|
||||
|
||||
## Referenz-Dateien
|
||||
|
||||
- `import-books.py` – Bestehende Importimplementierung (10 Klassiker)
|
||||
- `shared/src/main/kotlin/de/bollwerk/shared/model/ResourceDto.kt` – Datenmodell
|
||||
- `server/src/main/kotlin/de/bollwerk/server/routes/ResourceRoutes.kt` – API-Endpunkte
|
||||
- `server/src/main/kotlin/de/bollwerk/server/service/ResourceAnalyzer.kt` – Dateianalyse (epub/pdf/images)
|
||||
|
|
@ -8,7 +8,7 @@
|
|||
## Hosting & Hardware
|
||||
|
||||
| Eigenschaft | Wert |
|
||||
|--------------|-------------------------------------------|
|
||||
| ----------- | ----------------------------------------- |
|
||||
| Anbieter | 1984 Hosting (1984.is), Reykjavik, Island |
|
||||
| VPS-Name | vpshoxc2sc |
|
||||
| IP | `195.246.231.210` |
|
||||
|
|
@ -23,7 +23,7 @@
|
|||
## Betriebssystem
|
||||
|
||||
| Eigenschaft | Wert |
|
||||
|-------------|-----------------------------|
|
||||
| ----------- | ------------------------------ |
|
||||
| OS | Debian GNU/Linux 12 (Bookworm) |
|
||||
| Kernel | 6.1.0-30-amd64 |
|
||||
|
||||
|
|
@ -32,7 +32,7 @@
|
|||
## Installierte System-Software
|
||||
|
||||
| Paket / Dienst | Version | Installationsweg | Systemd-Service |
|
||||
|----------------|-------------|------------------|---------------------|
|
||||
| -------------- | --------------- | ------------------------------------- | ----------------- |
|
||||
| Docker CE | 29.5.0 | apt (Docker repo) | `docker.service` |
|
||||
| Docker Compose | v5.1.3 | docker-compose-plugin (apt) | – |
|
||||
| Caddy | v2.11.3 | apt (caddy repo) | `caddy.service` |
|
||||
|
|
@ -307,7 +307,7 @@ mail.bollwerk.online {
|
|||
## Netzwerk & Ports
|
||||
|
||||
| Port | Protokoll | Dienst | Erreichbarkeit |
|
||||
|------|-----------|---------------------|----------------|
|
||||
| ---- | --------- | ------------------ | --------------------- |
|
||||
| 22 | TCP | SSH | Extern |
|
||||
| 80 | TCP | Caddy (HTTP→HTTPS) | Extern |
|
||||
| 443 | TCP | Caddy (HTTPS) | Extern |
|
||||
|
|
|
|||
273
import-books-handwerk.py
Normal file
273
import-books-handwerk.py
Normal file
|
|
@ -0,0 +1,273 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Importiert 10 freie Handwerk-/Technik-Bücher (Project Gutenberg) in die Bollwerk-Ressourcen.
|
||||
Fokus: Nützliches Wissen für die Postapokalypse.
|
||||
|
||||
Verwendung:
|
||||
python import-books-handwerk.py
|
||||
python import-books-handwerk.py --dry-run
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import uuid
|
||||
import urllib.request
|
||||
import urllib.error
|
||||
|
||||
SERVER_URL = "https://bollwerk.online"
|
||||
|
||||
BOOKS = [
|
||||
{
|
||||
"gutenberg_id": 10136,
|
||||
"title": "The Book of Household Management",
|
||||
"author": "Mrs. Beeton",
|
||||
"description": "Das viktorianische Standardwerk für Haushaltsführung: Kochen, Konservieren, Reinigung, Krankenpflege, Tierhaltung, Budgetplanung. Ein Lexikon der praktischen Lebenskunst in 2000+ Seiten.",
|
||||
"tags": ["nachschlagewerk", "handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "1861-10-01",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 13510,
|
||||
"title": "Knots, Splices and Rope Work",
|
||||
"author": "A. Hyatt Verrill",
|
||||
"description": "Vollständige Anleitung zu Knoten, Spleißen und Seilwerk: über 100 Knoten mit Abbildungen. Unverzichtbar für Bauen, Sichern, Transportieren und Überleben ohne moderne Hilfsmittel.",
|
||||
"tags": ["handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "1917-01-01",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 47760,
|
||||
"title": "Three Hundred Things a Bright Boy Can Do",
|
||||
"author": "Various",
|
||||
"description": "Praktisches DIY-Kompendium: Elektrik, Mechanik, Holzarbeiten, Metallbearbeitung, Fotografie, Modellbau. 300 Projekte mit detaillierten Bauanleitungen – ein Maker-Handbuch aus dem frühen 20. Jahrhundert.",
|
||||
"tags": ["handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "1911-01-01",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 56601,
|
||||
"title": "A Text-book of Tanning",
|
||||
"author": "H. R. Procter",
|
||||
"description": "Wissenschaftliches Lehrbuch der Lederherstellung: Gerbstoffe, Hautbehandlung, pflanzliches und chemisches Gerben. Essentielles Handwerk für Kleidung, Schuhe und Ausrüstung ohne Industrie.",
|
||||
"tags": ["handbuch", "nachschlagewerk"],
|
||||
"language": "en",
|
||||
"release_date": "1885-01-01",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 67636,
|
||||
"title": "The Useful Arts Employed in the Construction of Dwelling Houses",
|
||||
"author": "Anonymous",
|
||||
"description": "Detailliertes Handbuch über alle Gewerke beim Hausbau: Mauerwerk, Zimmerei, Dachdecken, Klempnerei, Verglasung, Putz und Anstrich. Praxis-Referenz für den Wiederaufbau.",
|
||||
"tags": ["handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "1860-01-01",
|
||||
"edition": "Second Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 56776,
|
||||
"title": "Practical Hand Book of Gas, Oil and Steam Engines",
|
||||
"author": "John B. Rathbun",
|
||||
"description": "Praxishandbuch für Verbrennungs- und Dampfmotoren: Aufbau, Wartung, Reparatur, Fehlersuche. Wissen, um Energieerzeugung und Antriebstechnik ohne Stromnetz zu beherrschen.",
|
||||
"tags": ["handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "1916-01-01",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 21682,
|
||||
"title": "The Field and Garden Vegetables of America",
|
||||
"author": "Fearing Burr",
|
||||
"description": "Enzyklopädie mit fast 1100 Gemüsesorten: Beschreibung, Anbau, Ernte, Lagerung. Das umfassendste Nachschlagewerk für Selbstversorger-Gartenbau in der Public Domain.",
|
||||
"tags": ["nachschlagewerk"],
|
||||
"language": "en",
|
||||
"release_date": "1863-01-01",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 50451,
|
||||
"title": "A Boy's Workshop",
|
||||
"author": "Harry Craigin",
|
||||
"description": "Einführung in grundlegende Werkstattarbeiten: Holzverbindungen, Metallarbeiten, Werkzeugherstellung, einfache Maschinenbau-Projekte. Praktisches Handwerk von Grund auf erklärt.",
|
||||
"tags": ["handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "1894-01-01",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 53139,
|
||||
"title": "Maxims and Instructions for the Boiler Room",
|
||||
"author": "N. Hawkins",
|
||||
"description": "Praxiswissen für Dampfkesselbetrieb: Druckregelung, Brennstoffeffizienz, Sicherheit, Wartung. Schlüsselkompetenz für jede postapokalyptische Dampf-Infrastruktur.",
|
||||
"tags": ["handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "1903-01-01",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 65967,
|
||||
"title": "The History of Silk, Cotton, Linen, Wool, and Other Fibrous Substances",
|
||||
"author": "Clinton G. Gilroy",
|
||||
"description": "Geschichte und Technik der Textilherstellung: Spinnen, Weben, Färben von der Rohfaser zum fertigen Stoff. Grundlagenwissen, um ohne Textilindustrie Kleidung zu produzieren.",
|
||||
"tags": ["nachschlagewerk", "handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "1845-01-01",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
]
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
RESET = "\033[0m"
|
||||
GREEN = "\033[92m"
|
||||
YELLOW = "\033[93m"
|
||||
RED = "\033[91m"
|
||||
|
||||
def ok(msg): print(f"{GREEN}[OK] {msg}{RESET}", flush=True)
|
||||
def fail(msg): print(f"{RED}[!!] {msg}{RESET}", flush=True)
|
||||
def step(msg): print(f"\n{YELLOW}{msg}{RESET}", flush=True)
|
||||
def info(msg): print(f" {msg}", flush=True)
|
||||
|
||||
|
||||
def login(username: str, password: str) -> str:
|
||||
payload = json.dumps({"username": username, "password": password}).encode()
|
||||
req = urllib.request.Request(
|
||||
f"{SERVER_URL}/api/auth/login",
|
||||
data=payload,
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req) as resp:
|
||||
data = json.loads(resp.read())
|
||||
return data["accessToken"]
|
||||
except urllib.error.HTTPError as e:
|
||||
body = e.read().decode() if e.fp else ""
|
||||
fail(f"Login fehlgeschlagen: {e.code} {body}")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def download_epub(gutenberg_id: int) -> bytes:
|
||||
urls = [
|
||||
f"https://www.gutenberg.org/ebooks/{gutenberg_id}.epub.noimages",
|
||||
f"https://www.gutenberg.org/ebooks/{gutenberg_id}.epub3.images",
|
||||
f"https://www.gutenberg.org/cache/epub/{gutenberg_id}/pg{gutenberg_id}.epub",
|
||||
]
|
||||
for url in urls:
|
||||
try:
|
||||
req = urllib.request.Request(url, headers={"User-Agent": "BollwerkImporter/1.0"})
|
||||
with urllib.request.urlopen(req, timeout=60) as resp:
|
||||
data = resp.read()
|
||||
if len(data) > 1000:
|
||||
return data
|
||||
except (urllib.error.HTTPError, urllib.error.URLError):
|
||||
continue
|
||||
raise RuntimeError(f"Konnte Gutenberg #{gutenberg_id} nicht herunterladen")
|
||||
|
||||
|
||||
def upload_resource(token: str, book: dict, file_bytes: bytes) -> dict:
|
||||
guid = str(uuid.uuid4())
|
||||
now = int(time.time() * 1000)
|
||||
|
||||
metadata = {
|
||||
"guid": guid,
|
||||
"title": book["title"],
|
||||
"description": book["description"],
|
||||
"tags": book["tags"],
|
||||
"fileFormat": "epub",
|
||||
"mimeType": "application/epub+zip",
|
||||
"fileSize": len(file_bytes),
|
||||
"releaseDate": book.get("release_date"),
|
||||
"createdAt": now,
|
||||
"updatedAt": now,
|
||||
"author": book.get("author"),
|
||||
"language": book.get("language", "en"),
|
||||
"edition": book.get("edition"),
|
||||
"downloadUrl": "",
|
||||
}
|
||||
|
||||
boundary = f"----BollwerkBoundary{uuid.uuid4().hex[:16]}"
|
||||
body = bytearray()
|
||||
body += f"--{boundary}\r\n".encode()
|
||||
body += b'Content-Disposition: form-data; name="metadata"\r\n'
|
||||
body += b"Content-Type: application/json\r\n\r\n"
|
||||
body += json.dumps(metadata).encode()
|
||||
body += b"\r\n"
|
||||
body += f"--{boundary}\r\n".encode()
|
||||
body += f'Content-Disposition: form-data; name="file"; filename="{guid}.epub"\r\n'.encode()
|
||||
body += b"Content-Type: application/epub+zip\r\n\r\n"
|
||||
body += file_bytes
|
||||
body += b"\r\n"
|
||||
body += f"--{boundary}--\r\n".encode()
|
||||
|
||||
req = urllib.request.Request(
|
||||
f"{SERVER_URL}/api/admin/resources",
|
||||
data=bytes(body),
|
||||
headers={
|
||||
"Authorization": f"Bearer {token}",
|
||||
"Content-Type": f"multipart/form-data; boundary={boundary}",
|
||||
},
|
||||
method="POST",
|
||||
)
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=120) as resp:
|
||||
return json.loads(resp.read())
|
||||
except urllib.error.HTTPError as e:
|
||||
error_body = e.read().decode() if e.fp else ""
|
||||
raise RuntimeError(f"Upload fehlgeschlagen: {e.code} {error_body}")
|
||||
|
||||
|
||||
def main():
|
||||
dry_run = "--dry-run" in sys.argv
|
||||
count = len(BOOKS)
|
||||
|
||||
step(f"=== Bollwerk Bücher-Import ({count} Handwerk & Technik) ===")
|
||||
|
||||
if not dry_run:
|
||||
username = os.environ.get("BOLLWERK_ADMIN_USER", "")
|
||||
password = os.environ.get("BOLLWERK_ADMIN_PASS", "")
|
||||
if not username or not password:
|
||||
fail("Setze BOLLWERK_ADMIN_USER und BOLLWERK_ADMIN_PASS als Env-Vars")
|
||||
sys.exit(1)
|
||||
step("1/3 Login als Admin...")
|
||||
token = login(username, password)
|
||||
ok(f"Eingeloggt als '{username}'")
|
||||
else:
|
||||
token = ""
|
||||
info("DRY-RUN: Kein Login, kein Upload")
|
||||
|
||||
step("2/3 Bücher herunterladen...")
|
||||
downloads = []
|
||||
for i, book in enumerate(BOOKS, 1):
|
||||
info(f" [{i:2d}/{count}] {book['title']} (Gutenberg #{book['gutenberg_id']})...")
|
||||
try:
|
||||
data = download_epub(book["gutenberg_id"])
|
||||
downloads.append((book, data))
|
||||
ok(f" {book['title']} – {len(data) / 1024:.0f} KB")
|
||||
except RuntimeError as e:
|
||||
fail(f" {e}")
|
||||
|
||||
if not dry_run:
|
||||
step("3/3 Upload auf Bollwerk-Server...")
|
||||
success = 0
|
||||
for i, (book, data) in enumerate(downloads, 1):
|
||||
info(f" [{i:2d}/{len(downloads)}] {book['title']}...")
|
||||
try:
|
||||
result = upload_resource(token, book, data)
|
||||
ok(f" {book['title']} → guid={result['guid']}")
|
||||
success += 1
|
||||
except RuntimeError as e:
|
||||
fail(f" {e}")
|
||||
time.sleep(0.5)
|
||||
step(f"Fertig: {success}/{len(downloads)} Bücher erfolgreich importiert.")
|
||||
else:
|
||||
step(f"DRY-RUN abgeschlossen: {len(downloads)} Bücher heruntergeladen, 0 hochgeladen.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
272
import-books-prepper.py
Normal file
272
import-books-prepper.py
Normal file
|
|
@ -0,0 +1,272 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Importiert 10 freie Prepper-/Survival-Bücher (Project Gutenberg) in die Bollwerk-Ressourcen.
|
||||
|
||||
Verwendung:
|
||||
python import-books-prepper.py
|
||||
python import-books-prepper.py --dry-run
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import uuid
|
||||
import urllib.request
|
||||
import urllib.error
|
||||
|
||||
SERVER_URL = "https://bollwerk.online"
|
||||
|
||||
BOOKS = [
|
||||
{
|
||||
"gutenberg_id": 521,
|
||||
"title": "The Life and Adventures of Robinson Crusoe",
|
||||
"author": "Daniel Defoe",
|
||||
"description": "Der Urtext des Survival-Genres: 28 Jahre allein auf einer Insel. Crusoe baut Unterkunft, züchtet Getreide, fertigt Werkzeuge – ein Lehrbuch der Selbstversorgung in Romanform.",
|
||||
"tags": ["abenteuer", "handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "1719-04-25",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 205,
|
||||
"title": "Walden, and On The Duty Of Civil Disobedience",
|
||||
"author": "Henry David Thoreau",
|
||||
"description": "Thoreaus Experiment des einfachen Lebens in der Wildnis. Philosophie der Selbstversorgung, Konsumkritik und bewusstem Verzicht – das intellektuelle Fundament jeder Prepper-Bewegung.",
|
||||
"tags": ["nachschlagewerk", "handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "1854-08-09",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 28255,
|
||||
"title": "Shelters, Shacks and Shanties",
|
||||
"author": "Daniel Carter Beard",
|
||||
"description": "Praktische Anleitung zum Bau von Unterkünften aus Naturmaterialien – von einfachen Notschutzhütten bis zu Blockhäusern. Mit detaillierten Illustrationen und Schritt-für-Schritt-Anleitungen.",
|
||||
"tags": ["handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "1914-01-01",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 34607,
|
||||
"title": "Woodcraft and Camping",
|
||||
"author": "George Washington Sears",
|
||||
"description": "Klassiker der Outdoor-Literatur: Ausrüstung, Feuer machen, Lagerplatz wählen, Kochen im Freien, Kanufahren. Geschrieben von 'Nessmuk', dem Vater des Leichtgewicht-Campings.",
|
||||
"tags": ["handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "1884-01-01",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 15158,
|
||||
"title": "In Time of Emergency",
|
||||
"author": "United States Office of Civil Defense",
|
||||
"description": "Offizielles US-Handbuch für den Katastrophenfall (1968): Atomarer Angriff, Naturkatastrophen, Evakuierung, Strahlenschutz, Wasservorräte, Erste Hilfe. DAS Prepper-Dokument der Kalten-Krieg-Ära.",
|
||||
"tags": ["handbuch", "nachschlagewerk"],
|
||||
"language": "en",
|
||||
"release_date": "1968-01-01",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 44215,
|
||||
"title": "The Book of Camp-Lore and Woodcraft",
|
||||
"author": "Daniel Carter Beard",
|
||||
"description": "Umfassendes Nachschlagewerk für das Leben im Freien: Feuermachen ohne Streichhölzer, essbare Wildpflanzen, Orientierung, Fallen bauen, Signalgebung und Notfallmedizin.",
|
||||
"tags": ["handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "1920-01-01",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 34501,
|
||||
"title": "Wolf and Coyote Trapping",
|
||||
"author": "A. R. Harding",
|
||||
"description": "Detailliertes Praxishandbuch: Fallenbau, Köder, Fährten lesen, Pelzverarbeitung. Unverzichtbares Wissen für die Nahrungsbeschaffung und den Schutz von Nutztieren in der Wildnis.",
|
||||
"tags": ["handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "1909-01-01",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 49513,
|
||||
"title": "The Complete Herbal",
|
||||
"author": "Nicholas Culpeper",
|
||||
"description": "Umfassendes Kräuterbuch mit hunderten Heilpflanzen, ihren Eigenschaften und medizinischen Anwendungen. Seit 1653 die Referenz für Pflanzenheilkunde – unverzichtbar, wenn Apotheken fehlen.",
|
||||
"tags": ["nachschlagewerk", "handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "1653-01-01",
|
||||
"edition": "Original Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 9550,
|
||||
"title": "Manual of Gardening",
|
||||
"author": "L. H. Bailey",
|
||||
"description": "Vollständiges Garten-Handbuch: Bodenbearbeitung, Aussaat, Schädlingsbekämpfung, Obstbau, Gemüseanbau. Praktisches Wissen für die Selbstversorgung mit Nahrungsmitteln.",
|
||||
"tags": ["handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "1910-01-01",
|
||||
"edition": "Second Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 59977,
|
||||
"title": "Canning, Freezing, Storing Garden Produce",
|
||||
"author": "United States Department of Agriculture",
|
||||
"description": "USDA-Anleitung zur Haltbarmachung von Lebensmitteln: Einkochen, Einfrieren, Trocknen, Lagern. Überlebenswichtiges Wissen, um Ernteüberschüsse über den Winter zu retten.",
|
||||
"tags": ["handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "1965-01-01",
|
||||
"edition": "USDA Bulletin",
|
||||
},
|
||||
]
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
RESET = "\033[0m"
|
||||
GREEN = "\033[92m"
|
||||
YELLOW = "\033[93m"
|
||||
RED = "\033[91m"
|
||||
|
||||
def ok(msg): print(f"{GREEN}[OK] {msg}{RESET}", flush=True)
|
||||
def fail(msg): print(f"{RED}[!!] {msg}{RESET}", flush=True)
|
||||
def step(msg): print(f"\n{YELLOW}{msg}{RESET}", flush=True)
|
||||
def info(msg): print(f" {msg}", flush=True)
|
||||
|
||||
|
||||
def login(username: str, password: str) -> str:
|
||||
payload = json.dumps({"username": username, "password": password}).encode()
|
||||
req = urllib.request.Request(
|
||||
f"{SERVER_URL}/api/auth/login",
|
||||
data=payload,
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req) as resp:
|
||||
data = json.loads(resp.read())
|
||||
return data["accessToken"]
|
||||
except urllib.error.HTTPError as e:
|
||||
body = e.read().decode() if e.fp else ""
|
||||
fail(f"Login fehlgeschlagen: {e.code} {body}")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def download_epub(gutenberg_id: int) -> bytes:
|
||||
urls = [
|
||||
f"https://www.gutenberg.org/ebooks/{gutenberg_id}.epub.noimages",
|
||||
f"https://www.gutenberg.org/ebooks/{gutenberg_id}.epub3.images",
|
||||
f"https://www.gutenberg.org/cache/epub/{gutenberg_id}/pg{gutenberg_id}.epub",
|
||||
]
|
||||
for url in urls:
|
||||
try:
|
||||
req = urllib.request.Request(url, headers={"User-Agent": "BollwerkImporter/1.0"})
|
||||
with urllib.request.urlopen(req, timeout=60) as resp:
|
||||
data = resp.read()
|
||||
if len(data) > 1000:
|
||||
return data
|
||||
except (urllib.error.HTTPError, urllib.error.URLError):
|
||||
continue
|
||||
raise RuntimeError(f"Konnte Gutenberg #{gutenberg_id} nicht herunterladen")
|
||||
|
||||
|
||||
def upload_resource(token: str, book: dict, file_bytes: bytes) -> dict:
|
||||
guid = str(uuid.uuid4())
|
||||
now = int(time.time() * 1000)
|
||||
|
||||
metadata = {
|
||||
"guid": guid,
|
||||
"title": book["title"],
|
||||
"description": book["description"],
|
||||
"tags": book["tags"],
|
||||
"fileFormat": "epub",
|
||||
"mimeType": "application/epub+zip",
|
||||
"fileSize": len(file_bytes),
|
||||
"releaseDate": book.get("release_date"),
|
||||
"createdAt": now,
|
||||
"updatedAt": now,
|
||||
"author": book.get("author"),
|
||||
"language": book.get("language", "en"),
|
||||
"edition": book.get("edition"),
|
||||
"downloadUrl": "",
|
||||
}
|
||||
|
||||
boundary = f"----BollwerkBoundary{uuid.uuid4().hex[:16]}"
|
||||
body = bytearray()
|
||||
body += f"--{boundary}\r\n".encode()
|
||||
body += b'Content-Disposition: form-data; name="metadata"\r\n'
|
||||
body += b"Content-Type: application/json\r\n\r\n"
|
||||
body += json.dumps(metadata).encode()
|
||||
body += b"\r\n"
|
||||
body += f"--{boundary}\r\n".encode()
|
||||
body += f'Content-Disposition: form-data; name="file"; filename="{guid}.epub"\r\n'.encode()
|
||||
body += b"Content-Type: application/epub+zip\r\n\r\n"
|
||||
body += file_bytes
|
||||
body += b"\r\n"
|
||||
body += f"--{boundary}--\r\n".encode()
|
||||
|
||||
req = urllib.request.Request(
|
||||
f"{SERVER_URL}/api/admin/resources",
|
||||
data=bytes(body),
|
||||
headers={
|
||||
"Authorization": f"Bearer {token}",
|
||||
"Content-Type": f"multipart/form-data; boundary={boundary}",
|
||||
},
|
||||
method="POST",
|
||||
)
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=120) as resp:
|
||||
return json.loads(resp.read())
|
||||
except urllib.error.HTTPError as e:
|
||||
error_body = e.read().decode() if e.fp else ""
|
||||
raise RuntimeError(f"Upload fehlgeschlagen: {e.code} {error_body}")
|
||||
|
||||
|
||||
def main():
|
||||
dry_run = "--dry-run" in sys.argv
|
||||
count = len(BOOKS)
|
||||
|
||||
step(f"=== Bollwerk Bücher-Import ({count} Prepper / Survival) ===")
|
||||
|
||||
if not dry_run:
|
||||
username = os.environ.get("BOLLWERK_ADMIN_USER", "")
|
||||
password = os.environ.get("BOLLWERK_ADMIN_PASS", "")
|
||||
if not username or not password:
|
||||
fail("Setze BOLLWERK_ADMIN_USER und BOLLWERK_ADMIN_PASS als Env-Vars")
|
||||
sys.exit(1)
|
||||
step("1/3 Login als Admin...")
|
||||
token = login(username, password)
|
||||
ok(f"Eingeloggt als '{username}'")
|
||||
else:
|
||||
token = ""
|
||||
info("DRY-RUN: Kein Login, kein Upload")
|
||||
|
||||
step("2/3 Bücher herunterladen...")
|
||||
downloads = []
|
||||
for i, book in enumerate(BOOKS, 1):
|
||||
info(f" [{i:2d}/{count}] {book['title']} (Gutenberg #{book['gutenberg_id']})...")
|
||||
try:
|
||||
data = download_epub(book["gutenberg_id"])
|
||||
downloads.append((book, data))
|
||||
ok(f" {book['title']} – {len(data) / 1024:.0f} KB")
|
||||
except RuntimeError as e:
|
||||
fail(f" {e}")
|
||||
|
||||
if not dry_run:
|
||||
step("3/3 Upload auf Bollwerk-Server...")
|
||||
success = 0
|
||||
for i, (book, data) in enumerate(downloads, 1):
|
||||
info(f" [{i:2d}/{len(downloads)}] {book['title']}...")
|
||||
try:
|
||||
result = upload_resource(token, book, data)
|
||||
ok(f" {book['title']} → guid={result['guid']}")
|
||||
success += 1
|
||||
except RuntimeError as e:
|
||||
fail(f" {e}")
|
||||
time.sleep(0.5)
|
||||
step(f"Fertig: {success}/{len(downloads)} Bücher erfolgreich importiert.")
|
||||
else:
|
||||
step(f"DRY-RUN abgeschlossen: {len(downloads)} Bücher heruntergeladen, 0 hochgeladen.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
287
import-books-references.py
Normal file
287
import-books-references.py
Normal file
|
|
@ -0,0 +1,287 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Importiert 10 freie Nachschlagewerke/Referenzwerke (Project Gutenberg) in die Bollwerk-Ressourcen.
|
||||
|
||||
Voraussetzungen:
|
||||
- Server läuft unter SERVER_URL
|
||||
- Admin-Credentials als Env-Vars: BOLLWERK_ADMIN_USER, BOLLWERK_ADMIN_PASS
|
||||
|
||||
Verwendung:
|
||||
python import-books-references.py
|
||||
python import-books-references.py --dry-run (nur Download, kein Upload)
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import uuid
|
||||
import urllib.request
|
||||
import urllib.error
|
||||
|
||||
SERVER_URL = "https://bollwerk.online"
|
||||
|
||||
# --- 10 ausgewählte Referenzwerke (Project Gutenberg, Public Domain) ---
|
||||
BOOKS = [
|
||||
{
|
||||
"gutenberg_id": 1497,
|
||||
"title": "The Republic",
|
||||
"author": "Plato",
|
||||
"description": "Platons Dialog über Gerechtigkeit, den idealen Staat und die Philosophenherrschaft. Grundlagenwerk der politischen Philosophie und Gesellschaftstheorie seit über 2400 Jahren.",
|
||||
"tags": ["nachschlagewerk"],
|
||||
"language": "en",
|
||||
"release_date": "-375",
|
||||
"edition": "Benjamin Jowett Translation",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 1228,
|
||||
"title": "On the Origin of Species",
|
||||
"author": "Charles Darwin",
|
||||
"description": "Darwins revolutionäre Theorie der natürlichen Selektion. Das Buch, das unser Verständnis des Lebens grundlegend veränderte – Pflichtlektüre für Naturwissenschaft und Evolutionsbiologie.",
|
||||
"tags": ["nachschlagewerk"],
|
||||
"language": "en",
|
||||
"release_date": "1859-11-24",
|
||||
"edition": "First Edition, 1859",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 3300,
|
||||
"title": "The Wealth of Nations",
|
||||
"author": "Adam Smith",
|
||||
"description": "Das Gründungswerk der modernen Ökonomie. Smith analysiert Arbeitsteilung, freie Märkte und die 'unsichtbare Hand' – bis heute Referenz für wirtschaftliches Denken.",
|
||||
"tags": ["nachschlagewerk", "handbuch"],
|
||||
"language": "en",
|
||||
"release_date": "1776-03-09",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 147,
|
||||
"title": "Common Sense",
|
||||
"author": "Thomas Paine",
|
||||
"description": "Die einflussreichste politische Flugschrift der amerikanischen Revolution. Paines klare Argumente für Unabhängigkeit und Selbstbestimmung inspirierten eine ganze Nation zum Handeln.",
|
||||
"tags": ["nachschlagewerk"],
|
||||
"language": "en",
|
||||
"release_date": "1776-01-10",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 4363,
|
||||
"title": "Beyond Good and Evil",
|
||||
"author": "Friedrich Nietzsche",
|
||||
"description": "Nietzsches Frontalangriff auf die traditionelle Philosophie. Hinterfragt Moral, Wahrheit und den Willen zur Macht – ein Schlüsseltext der modernen Philosophie.",
|
||||
"tags": ["nachschlagewerk"],
|
||||
"language": "en",
|
||||
"release_date": "1886-01-01",
|
||||
"edition": "Helen Zimmern Translation, 1906",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 3207,
|
||||
"title": "Leviathan",
|
||||
"author": "Thomas Hobbes",
|
||||
"description": "Hobbes' Theorie des Gesellschaftsvertrags und der absoluten Staatsgewalt. Geschrieben im englischen Bürgerkrieg – das Standardwerk über Macht, Ordnung und den Naturzustand des Menschen.",
|
||||
"tags": ["nachschlagewerk"],
|
||||
"language": "en",
|
||||
"release_date": "1651-04-01",
|
||||
"edition": "First Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 1404,
|
||||
"title": "The Federalist Papers",
|
||||
"author": "Alexander Hamilton, James Madison, John Jay",
|
||||
"description": "85 Essays zur Verteidigung der US-Verfassung. Das wichtigste Dokument amerikanischer Staatstheorie – Referenzwerk für Gewaltenteilung, Föderalismus und demokratische Institutionen.",
|
||||
"tags": ["nachschlagewerk"],
|
||||
"language": "en",
|
||||
"release_date": "1788-01-01",
|
||||
"edition": "First Collected Edition",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 1998,
|
||||
"title": "Thus Spake Zarathustra",
|
||||
"author": "Friedrich Nietzsche",
|
||||
"description": "Nietzsches philosophisches Hauptwerk in poetischer Form. Einführung des Übermenschen, des Willens zur Macht und der ewigen Wiederkehr – eine der einflussreichsten Schriften der Moderne.",
|
||||
"tags": ["nachschlagewerk"],
|
||||
"language": "en",
|
||||
"release_date": "1885-01-01",
|
||||
"edition": "Thomas Common Translation, 1909",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 61,
|
||||
"title": "The Communist Manifesto",
|
||||
"author": "Karl Marx, Friedrich Engels",
|
||||
"description": "Das Manifest der Kommunistischen Partei – die wohl wirkungsmächtigste politische Schrift des 19. Jahrhunderts. Analyse des Klassenkampfs und Vision einer klassenlosen Gesellschaft.",
|
||||
"tags": ["nachschlagewerk"],
|
||||
"language": "en",
|
||||
"release_date": "1848-02-21",
|
||||
"edition": "Samuel Moore Translation, 1888",
|
||||
},
|
||||
{
|
||||
"gutenberg_id": 4280,
|
||||
"title": "The Critique of Pure Reason",
|
||||
"author": "Immanuel Kant",
|
||||
"description": "Kants Meisterwerk über die Grenzen menschlicher Erkenntnis. Untersucht, was wir wissen können, bevor wir es erfahren – Grundlage der gesamten modernen Erkenntnistheorie.",
|
||||
"tags": ["nachschlagewerk"],
|
||||
"language": "en",
|
||||
"release_date": "1781-01-01",
|
||||
"edition": "J. M. D. Meiklejohn Translation, 1855",
|
||||
},
|
||||
]
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
RESET = "\033[0m"
|
||||
GREEN = "\033[92m"
|
||||
YELLOW = "\033[93m"
|
||||
RED = "\033[91m"
|
||||
|
||||
def ok(msg): print(f"{GREEN}[OK] {msg}{RESET}", flush=True)
|
||||
def fail(msg): print(f"{RED}[!!] {msg}{RESET}", flush=True)
|
||||
def step(msg): print(f"\n{YELLOW}{msg}{RESET}", flush=True)
|
||||
def info(msg): print(f" {msg}", flush=True)
|
||||
|
||||
|
||||
def login(username: str, password: str) -> str:
|
||||
"""Login and return JWT access token."""
|
||||
payload = json.dumps({"username": username, "password": password}).encode()
|
||||
req = urllib.request.Request(
|
||||
f"{SERVER_URL}/api/auth/login",
|
||||
data=payload,
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req) as resp:
|
||||
data = json.loads(resp.read())
|
||||
return data["accessToken"]
|
||||
except urllib.error.HTTPError as e:
|
||||
body = e.read().decode() if e.fp else ""
|
||||
fail(f"Login fehlgeschlagen: {e.code} {body}")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def download_epub(gutenberg_id: int) -> bytes:
|
||||
"""Download ePub from Project Gutenberg (prefer lightweight no-images version)."""
|
||||
urls = [
|
||||
f"https://www.gutenberg.org/ebooks/{gutenberg_id}.epub.noimages",
|
||||
f"https://www.gutenberg.org/ebooks/{gutenberg_id}.epub3.images",
|
||||
f"https://www.gutenberg.org/cache/epub/{gutenberg_id}/pg{gutenberg_id}.epub",
|
||||
]
|
||||
for url in urls:
|
||||
try:
|
||||
req = urllib.request.Request(url, headers={"User-Agent": "BollwerkImporter/1.0"})
|
||||
with urllib.request.urlopen(req, timeout=60) as resp:
|
||||
data = resp.read()
|
||||
if len(data) > 1000: # sanity check
|
||||
return data
|
||||
except (urllib.error.HTTPError, urllib.error.URLError):
|
||||
continue
|
||||
raise RuntimeError(f"Konnte Gutenberg #{gutenberg_id} nicht herunterladen")
|
||||
|
||||
|
||||
def upload_resource(token: str, book: dict, file_bytes: bytes) -> dict:
|
||||
"""Upload resource via multipart POST to admin API."""
|
||||
guid = str(uuid.uuid4())
|
||||
now = int(time.time() * 1000)
|
||||
|
||||
metadata = {
|
||||
"guid": guid,
|
||||
"title": book["title"],
|
||||
"description": book["description"],
|
||||
"tags": book["tags"],
|
||||
"fileFormat": "epub",
|
||||
"mimeType": "application/epub+zip",
|
||||
"fileSize": len(file_bytes),
|
||||
"releaseDate": book.get("release_date"),
|
||||
"createdAt": now,
|
||||
"updatedAt": now,
|
||||
"author": book.get("author"),
|
||||
"language": book.get("language", "en"),
|
||||
"edition": book.get("edition"),
|
||||
"downloadUrl": "",
|
||||
}
|
||||
|
||||
boundary = f"----BollwerkBoundary{uuid.uuid4().hex[:16]}"
|
||||
body = bytearray()
|
||||
|
||||
# Part 1: metadata JSON
|
||||
body += f"--{boundary}\r\n".encode()
|
||||
body += b'Content-Disposition: form-data; name="metadata"\r\n'
|
||||
body += b"Content-Type: application/json\r\n\r\n"
|
||||
body += json.dumps(metadata).encode()
|
||||
body += b"\r\n"
|
||||
|
||||
# Part 2: file
|
||||
body += f"--{boundary}\r\n".encode()
|
||||
body += f'Content-Disposition: form-data; name="file"; filename="{guid}.epub"\r\n'.encode()
|
||||
body += b"Content-Type: application/epub+zip\r\n\r\n"
|
||||
body += file_bytes
|
||||
body += b"\r\n"
|
||||
|
||||
# End boundary
|
||||
body += f"--{boundary}--\r\n".encode()
|
||||
|
||||
req = urllib.request.Request(
|
||||
f"{SERVER_URL}/api/admin/resources",
|
||||
data=bytes(body),
|
||||
headers={
|
||||
"Authorization": f"Bearer {token}",
|
||||
"Content-Type": f"multipart/form-data; boundary={boundary}",
|
||||
},
|
||||
method="POST",
|
||||
)
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=120) as resp:
|
||||
return json.loads(resp.read())
|
||||
except urllib.error.HTTPError as e:
|
||||
error_body = e.read().decode() if e.fp else ""
|
||||
raise RuntimeError(f"Upload fehlgeschlagen: {e.code} {error_body}")
|
||||
|
||||
|
||||
def main():
|
||||
dry_run = "--dry-run" in sys.argv
|
||||
|
||||
step("=== Bollwerk Bücher-Import (10 Nachschlagewerke / Referenzwerke) ===")
|
||||
|
||||
if not dry_run:
|
||||
username = os.environ.get("BOLLWERK_ADMIN_USER", "")
|
||||
password = os.environ.get("BOLLWERK_ADMIN_PASS", "")
|
||||
if not username or not password:
|
||||
fail("Setze BOLLWERK_ADMIN_USER und BOLLWERK_ADMIN_PASS als Env-Vars")
|
||||
sys.exit(1)
|
||||
|
||||
step("1/3 Login als Admin...")
|
||||
token = login(username, password)
|
||||
ok(f"Eingeloggt als '{username}'")
|
||||
else:
|
||||
token = ""
|
||||
info("DRY-RUN: Kein Login, kein Upload")
|
||||
|
||||
step("2/3 Bücher herunterladen...")
|
||||
downloads = []
|
||||
for i, book in enumerate(BOOKS, 1):
|
||||
info(f" [{i:2d}/10] {book['title']} (Gutenberg #{book['gutenberg_id']})...")
|
||||
try:
|
||||
data = download_epub(book["gutenberg_id"])
|
||||
downloads.append((book, data))
|
||||
ok(f" {book['title']} – {len(data) / 1024:.0f} KB")
|
||||
except RuntimeError as e:
|
||||
fail(f" {e}")
|
||||
|
||||
if not dry_run:
|
||||
step("3/3 Upload auf Bollwerk-Server...")
|
||||
success = 0
|
||||
for i, (book, data) in enumerate(downloads, 1):
|
||||
info(f" [{i:2d}/{len(downloads)}] {book['title']}...")
|
||||
try:
|
||||
result = upload_resource(token, book, data)
|
||||
ok(f" {book['title']} → guid={result['guid']}")
|
||||
success += 1
|
||||
except RuntimeError as e:
|
||||
fail(f" {e}")
|
||||
time.sleep(0.5) # rate limiting
|
||||
|
||||
step(f"Fertig: {success}/{len(downloads)} Bücher erfolgreich importiert.")
|
||||
else:
|
||||
step(f"DRY-RUN abgeschlossen: {len(downloads)} Bücher heruntergeladen, 0 hochgeladen.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -14,7 +14,7 @@ bollwerk {
|
|||
jwtIssuer = "bollwerk"
|
||||
jwtAudience = "bollwerk-client"
|
||||
accessTokenExpiryMs = 3600000
|
||||
refreshTokenExpiryMs = 2592000000
|
||||
refreshTokenExpiryMs = 15552000000
|
||||
appVersionCode = 1
|
||||
appVersionCode = ${?BOLLWERK_APP_VERSION_CODE}
|
||||
appVersionName = "1.0.0"
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ internal fun testMapConfig(vararg extra: Pair<String, String>) = listOf(
|
|||
"bollwerk.jwtIssuer" to TEST_JWT_ISSUER,
|
||||
"bollwerk.jwtAudience" to TEST_JWT_AUDIENCE,
|
||||
"bollwerk.accessTokenExpiryMs" to "3600000",
|
||||
"bollwerk.refreshTokenExpiryMs" to "2592000000",
|
||||
"bollwerk.refreshTokenExpiryMs" to "15552000000",
|
||||
"bollwerk.appVersionCode" to "42",
|
||||
"bollwerk.appVersionName" to "2.1.0",
|
||||
"bollwerk.adminToken" to TEST_ADMIN_TOKEN
|
||||
|
|
|
|||
Loading…
Reference in a new issue