M29 只解决了「回书架不重扫」;首次进书架仍要全量扫 434 目录(真机 3.0–3.4s, 其中 opendir+readdir 占 96%)。本单把扫描结果 + 每个扫描目录的 mtime 存到 MeowData/os.meow.reader/index/shelf.idx,冷启动逐目录 stat 比对 mtime,命中即 完全跳过 readdir;任何异常一律 fail-open 全量重扫。 先量再选(任务 1):宿主 434 目录/595 本/51 rar 虚拟库(贴真机量级) - 434 次目录 stat = 0.48ms/pass(1.1µs/次,不含 readdir); - 读 72KB 索引 = 0.009ms;全量扫描 3.54ms;索引命中 1.29ms(ts_fn=NULL)。 真机按 M28 实测 1079 次 stat=160–170ms(≈150µs/次)外推:434 次 ≈ 65ms, 远低于「可能不可行」的 1s 阈值 → 选「校验全部扫描目录 mtime」(非只 level-1)。 strace 断言:indexonly 载入 0 次 getdents64、恰好 434 次 stat(真的没 readdir)。 生产路径 ≈ 65ms(stat) + <20ms(索引读) + 40–50ms(rdprog×595) + <20ms(build_rows) ≈ 0.13–0.15s(对比 3.0s)。 索引格式 v1(手工小端序列化,不依赖结构体布局): header(magic/version/header_size/root_len) + root + dir 记录×(rel,mtime_s, mtime_ns,books_here) + book 记录×(rel,size,fmt) + trailer(ndirs,nbooks,flags, depth_skipped,crc32,magic)。truncated/depth_skipped 存在 flags/字段里,载入时 如实重建提示条;**不存阅读进度 ts**,载入后每本由 rdprog_load 现读再排序。 失效语义(存每个扫描过的目录,不只 level-1): - level-2 新增 / level-3 深处新增 / 删书 → 该目录 mtime 变 → 重扫; - 一级目录改名 → 父目录 mtime 变 + 旧目录 stat ENOENT → 重扫; - 超深(>6 层)未扫描子树不在索引内,其内部变化不自动失效(本来就不显示), 点「刷新」更新 depth_skipped 计数;mtime 精度受限(exFAT 10ms/FAT32 2s)的 同刻度变更也可能漏判,点「刷新」必可纠正。 M29 语义不变:首次进书架=索引或全扫;回书架=不扫;刷新=删索引+全扫(之后原子 写回新索引)。 安全/健壮(宁可慢不可错): - 写:shelf.idx.tmp 全量写完(尽力 fsync)后 rename;中途被杀只剩 tmp,正式索引 要么旧要么完整新;rename 覆盖失败先删旧再改名;fflush/写错即放弃,绝不影响书架。 - 读:O_NOFOLLOW|O_NONBLOCK + S_ISREG(FIFO/符号链接直接拒绝不阻塞);CRC32 覆盖 全部记录;rel 做绝对路径/“..”/内嵌 NUL/长度校验;版本/根路径/计数/逐目录 mtime 任一不符即重扫;载入崩溃路径全部 goto 清理,不改书架可用性。 - writer_open 失败(卡满/写保护/无 MeowData)→ 扫描不采集目录、照常出书; commit 失败(含只读目录 rename 失败)不留正式索引。 测试:test/shelf_index_smoke 154 pass(含四种变更场景失效、截断/字节翻转/垃圾/ 缺失/FIFO/符号链接/版本不符/根路径不符、ts 现读、truncated/depth_skipped 重建、 只读目录写失败、空 Books、root 缺失、.tmp 残渣),ASan+UBSan 干净; test/shelf_index_bench 模拟 434 目录/595 本/51 rar:scan/load 均 595 本、 truncated=0/depth_skipped=0、索引 72,156B。build_reader.sh 全绿 0 warning (core 6846 / seg 19 / chapter 71 / shelf_group 63 / index 154 / font 14 / loaders PASS)。shelf_bench_ondev 增加 index_load/scan_save 计时与 MEOW_BENCH_IDX=miss(真机首开命中 vs 未命中对比,待有卡复核)。 交付:app.json 0.1.4/code5 → 0.1.5/code6;reader md5 df4cadcf 1,711,924B;mpk md5 fec7b1bb 814,535B;未签名不推卡。
197 lines
12 KiB
Bash
Executable File
197 lines
12 KiB
Bash
Executable File
#!/bin/bash
|
||
# build_reader.sh — os.meow.reader R2 可复现构建:
|
||
# vendor: freetype(静态)/miniz/libmobi/stb/zlib 头 → 交叉 mipsel 编 reader 二进制
|
||
# + 宿主 core 单测(R1+loaders) + qemu-mipsel 冒烟(rdft/book/formats)。
|
||
#
|
||
# 产物:
|
||
# reader mipsel 静态 ELF(链 meow_ui.a + freetype + miniz + libmobi + zlib)
|
||
# test/test_reader_core_host 宿主 core 单测(enc/layout/txt)
|
||
# test/shelf_group_smoke_host 宿主书架分组单测(M27/M31 副题)
|
||
# test/shelf_index_smoke_host 宿主书架索引单测(M32:命中/失效/fail-open/ts 不缓存)
|
||
# test/shelf_index_bench_host 宿主索引耗时基准(M32,只编不跑;bench/indexonly)
|
||
# test/font_resolve_smoke_host 宿主字体路径记忆化单测(M31)
|
||
# test/font_resolve_bench_host 宿主字体 walk/命中耗时对比(M31,只编不跑)
|
||
# test/loaders_smoke_host 宿主四格式 loader 单测
|
||
# test/rdft_smoke_mipsel qemu-mipsel FreeType 冒烟
|
||
# test/book_smoke_mipsel qemu-mipsel TXT 书桥冒烟
|
||
# test/formats_smoke_mipsel qemu-mipsel 四格式书桥冒烟
|
||
#
|
||
# 约束:只写本目录(patch_v2/reader_app/);不改 sdk/;-Wall -Wextra 0 warning(自有代码)。
|
||
# vendor/(第三方源码+构建产物)gitignore,不入仓;build 脚本可复现。
|
||
set -e
|
||
cd "$(dirname "$0")"
|
||
ROOT="$HOME/Project/R3ProII"
|
||
CC="$ROOT/.tools/zig-mipsel-cc"
|
||
SYSROOT="$ROOT/.tools/mipsel-linux-muslsf-cross/mipsel-linux-muslsf"
|
||
HOST_CC="${HOST_CC:-gcc}"
|
||
VENDOR="$PWD/vendor"
|
||
OBJ="$VENDOR/build"
|
||
# qemu-mipsel 运行时冒烟:宿主没装 qemu(如 Termux 无 mipsel user-mode 包)时,
|
||
# 仍编译 mipsel 冒烟二进制,只跳过执行;设 QEMU_BIN=<路径> 可指定其它模拟器。
|
||
QEMU_BIN="${QEMU_BIN:-qemu-mipsel}"
|
||
run_qemu() {
|
||
if [ "${SKIP_QEMU:-0}" = 1 ] || ! command -v "$QEMU_BIN" >/dev/null 2>&1; then
|
||
shift
|
||
echo " SKIP qemu 执行(未找到 $QEMU_BIN):$*"
|
||
return 0
|
||
fi
|
||
timeout "$@"
|
||
}
|
||
mkdir -p "$OBJ"
|
||
|
||
# ---------- 1) vendor 拉取/对齐 ----------
|
||
FT_VER="2.14.3"; MINIZ_URL="https://codeload.github.com/richgel999/miniz/tar.gz/refs/heads/master"
|
||
# libmobi 版本锁定:见 NOTICE「Locked version (commit)」。LGPL-3.0 §4(d) 要求「已分发二进制
|
||
# 对应的 libmobi 源码」与公开源码一致可复现,故不用 refs/heads/master(移动分支),改锁到精确
|
||
# commit SHA(2024-10-29 的 9062742...,与上级代理 agent-332 复验/NOTICE 声明一致,已验证可拉取)。
|
||
LM_URL="https://codeload.github.com/bfabiszewski/libmobi/tar.gz/906274205c11944b628da1c553b255acb1af7c55"
|
||
mkdir -p "$VENDOR"
|
||
fetch_freetype() {
|
||
if [ -n "$FREETYPE_SRC" ] && [ -f "$FREETYPE_SRC/CMakeLists.txt" ]; then cp -r "$FREETYPE_SRC" vendor/freetype-src;
|
||
elif [ ! -d "$VENDOR/freetype-src" ] && [ -f /tmp/freetype/CMakeLists.txt ]; then cp -r /tmp/freetype "$VENDOR/freetype-src";
|
||
elif [ ! -d "$VENDOR/freetype-src" ]; then
|
||
( cd "$VENDOR" && curl -fsSL -o ft.tar.xz "https://download.savannah.gnu.org/releases/freetype/freetype-$FT_VER.tar.xz" \
|
||
&& mkdir -p freetype-src && tar xf ft.tar.xz -C freetype-src --strip-components=1 && rm ft.tar.xz )
|
||
fi
|
||
[ -f "$VENDOR/freetype-src/CMakeLists.txt" ] || { echo "freetype 不可用"; exit 1; }
|
||
}
|
||
fetch_miniz() {
|
||
if [ ! -f "$VENDOR/miniz/miniz.c" ]; then
|
||
( cd /tmp && curl -fsSL -o miniz.tgz "$MINIZ_URL" && mkdir -p miniz_m && tar xzf miniz.tgz -C miniz_m --strip-components=1 \
|
||
&& cp miniz_m/miniz.c miniz_m/miniz.h miniz_m/miniz_common.h miniz_m/miniz_tdef.c miniz_m/miniz_tdef.h \
|
||
miniz_m/miniz_tinfl.c miniz_m/miniz_tinfl.h miniz_m/miniz_zip.c miniz_m/miniz_zip.h "$VENDOR/miniz/" )
|
||
fi
|
||
[ -f "$VENDOR/miniz/miniz.c" ] || { echo "miniz 不可用"; exit 1; }
|
||
}
|
||
fetch_libmobi() {
|
||
# 若 vendor/libmobi 来自旧版(LM_URL 改 pin 前抓的 master),请先删 vendor/libmobi 再重跑,
|
||
# 以确保取到下方锁定的 commit。此处仅在其缺失时拉取。
|
||
if [ ! -f "$VENDOR/libmobi/src/mobi.h" ]; then
|
||
( cd /tmp && curl -fsSL -o lm.tgz "$LM_URL" && mkdir -p lm_m && tar xzf lm.tgz -C lm_m --strip-components=1 \
|
||
&& mkdir -p "$VENDOR/libmobi" && cp -r lm_m/src "$VENDOR/libmobi/src" && cp lm_m/COPYING "$VENDOR/libmobi/COPYING" )
|
||
fi
|
||
[ -f "$VENDOR/libmobi/src/mobi.h" ] || { echo "libmobi 不可用"; exit 1; }
|
||
}
|
||
fetch_stb() { [ -f "$VENDOR/stb/stb_image.h" ] || { mkdir -p "$VENDOR/stb"; cp "$ROOT/patch_v2/sdk/stb_image.h" "$VENDOR/stb/stb_image.h"; }; }
|
||
fetch_freetype; fetch_miniz; fetch_libmobi; fetch_stb
|
||
if [ ! -f "$VENDOR/zlib/zlib.h" ]; then mkdir -p "$VENDOR/zlib"; cp /usr/include/zlib.h /usr/include/zconf.h "$VENDOR/zlib/" 2>/dev/null || true; fi
|
||
|
||
# ---------- 2) FreeType mipsel 静态构建 ----------
|
||
FT_SRC="$VENDOR/freetype-src"; FT_BUILD="$VENDOR/freetype-build"
|
||
if [ ! -f "$FT_BUILD/libfreetype.a" ] || [ -n "$FORCE_FT" ]; then
|
||
rm -rf "$FT_BUILD"
|
||
cmake -S "$FT_SRC" -B "$FT_BUILD" -DCMAKE_SYSTEM_NAME=Linux -DCMAKE_C_COMPILER="$CC" \
|
||
-DCMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF \
|
||
-DFT_DISABLE_ZLIB=TRUE -DFT_DISABLE_BZIP2=TRUE -DFT_DISABLE_PNG=TRUE -DFT_DISABLE_HARFBUZZ=TRUE -DFT_DISABLE_BROTLI=TRUE >/dev/null
|
||
cmake --build "$FT_BUILD" -j"$(nproc)" >/dev/null 2>&1
|
||
fi
|
||
FT_INCS="-I$FT_BUILD -I$FT_BUILD/include -I$FT_SRC/include"
|
||
|
||
COREINC="-Icore -Ivendor/miniz -Ivendor/stb -Ivendor/libmobi/src -Ivendor/zlib"
|
||
MZFLAGS="-DMINIZ_NO_ZLIB_COMPATIBLE_NAMES -DMINIZ_NO_TIME"
|
||
|
||
# ---------- 3) miniz + libmobi 对象(mipsel) ----------
|
||
build_mipsel_libs() {
|
||
for f in vendor/miniz/miniz_tinfl.c vendor/miniz/miniz_tdef.c vendor/miniz/miniz_zip.c vendor/miniz/miniz.c; do
|
||
b="$(basename "$f" .c)"; [ -f "$OBJ/mz_$b.o" ] || $CC -Os -static -Wall -Wextra -Ivendor/miniz $MZFLAGS -c "$f" -o "$OBJ/mz_$b.o"
|
||
done
|
||
for f in vendor/libmobi/src/*.c; do b="$(basename "$f" .c)"; [ "$b" = "miniz" ] && continue
|
||
[ -f "$OBJ/lm_$b.o" ] || $CC -Os -static -w -Ivendor/libmobi/src -Ivendor/zlib -c "$f" -o "$OBJ/lm_$b.o"
|
||
done
|
||
# stb_image 的实现由 meow_ui.a 提供(SDK 自 M6.45 收编 stb_image.c.o);此处只编封装,
|
||
# 否则链接 meow_ui.a 时 stbi_* 会与归档内 stb_image.o 重复定义。
|
||
[ -f "$OBJ/rdimg.o" ] || $CC -Os -static -Wall -Wextra -Ivendor/stb -c engine/rdimg.c -o "$OBJ/rdimg.o"
|
||
}
|
||
build_mipsel_libs
|
||
|
||
# ---------- 4) 宿主 loader 单测(miniz/libmobi 用宿主编一遍) ----------
|
||
echo "==> host core test (R1)"
|
||
$HOST_CC -O2 -Wall -Wextra -Icore -I. test/test_reader_core.c core/rdenc.c core/rdlayout.c core/rdtxt.c -o test/test_reader_core_host -lm
|
||
./test/test_reader_core_host
|
||
|
||
echo "==> host seg test (R3.2)"
|
||
$HOST_CC -O2 -Wall -Wextra -Icore test/seg_smoke.c core/seg.c core/rdenc.c -o test/seg_smoke_host -lm
|
||
./test/seg_smoke_host
|
||
|
||
echo "==> host chapter test (R3.3)"
|
||
$HOST_CC -O2 -Wall -Wextra -Icore test/chapter_smoke.c core/rdchapter.c core/seg.c core/rdenc.c -o test/chapter_smoke_host -lm
|
||
./test/chapter_smoke_host
|
||
|
||
echo "==> host shelf group test (M27)"
|
||
$HOST_CC -O2 -Wall -Wextra -Iapp -I../sdk test/shelf_group_smoke.c app/shelf_model.c app/progress.c app/reader_conf.c -o test/shelf_group_smoke_host
|
||
./test/shelf_group_smoke_host
|
||
|
||
echo "==> host shelf index test (M32)"
|
||
$HOST_CC -O2 -Wall -Wextra -Iapp -I../sdk test/shelf_index_smoke.c app/shelf_model.c app/shelf_index.c app/progress.c -o test/shelf_index_smoke_host
|
||
./test/shelf_index_smoke_host
|
||
$HOST_CC -O2 -Wall -Wextra -Iapp -I../sdk test/shelf_index_bench.c app/shelf_model.c app/shelf_index.c -o test/shelf_index_bench_host
|
||
|
||
echo "==> host font resolve test (M31)"
|
||
$HOST_CC -O2 -Wall -Wextra -Iapp test/font_resolve_smoke.c app/font_resolve.c -o test/font_resolve_smoke_host
|
||
./test/font_resolve_smoke_host
|
||
$HOST_CC -O2 -Wall -Wextra -Iapp test/font_resolve_bench.c app/font_resolve.c -o test/font_resolve_bench_host
|
||
|
||
echo "==> host shelf scan bench (M27,只编不跑;用法 ./test/shelf_scan_bench_host <Books目录> [progress|noop] [轮数])"
|
||
$HOST_CC -O2 -Wall -Wextra -Iapp -I../sdk test/shelf_scan_bench.c app/shelf_model.c app/progress.c -o test/shelf_scan_bench_host
|
||
|
||
echo "==> host loaders test"
|
||
HOBJ="$OBJ/host"; mkdir -p "$HOBJ"
|
||
for f in vendor/miniz/miniz_tinfl.c vendor/miniz/miniz_tdef.c vendor/miniz/miniz_zip.c vendor/miniz/miniz.c; do
|
||
b="$(basename "$f" .c)"; [ -f "$HOBJ/mz_$b.o" ] || $HOST_CC -O2 -Ivendor/miniz $MZFLAGS -c "$f" -o "$HOBJ/mz_$b.o"
|
||
done
|
||
for f in vendor/libmobi/src/*.c; do b="$(basename "$f" .c)"; [ "$b" = "miniz" ] && continue
|
||
[ -f "$HOBJ/lm_$b.o" ] || $HOST_CC -O2 -w -Ivendor/libmobi/src -Ivendor/zlib -c "$f" -o "$HOBJ/lm_$b.o"
|
||
done
|
||
HCORE=""
|
||
for f in rdxml rdhtml rdsb rdzip rdsrc rdepub rdfb2 rdmobi rdcbz rdenc rdtxt; do
|
||
$HOST_CC -O2 -Wall -Wextra $COREINC -c core/$f.c -o "$HOBJ/core_$f.o"
|
||
done
|
||
python3 tools/gen_samples.py >/dev/null
|
||
$HOST_CC -O2 -Wall -Wextra $COREINC test/loaders_smoke.c "$HOBJ"/*.o -lz -lm -o test/loaders_smoke_host
|
||
./test/loaders_smoke_host /tmp/rd_samples
|
||
|
||
# ---------- 5) qemu-mipsel FreeType 冒烟 ----------
|
||
echo "==> mipsel ft smoke"
|
||
$CC -Os -static -Wall -Wextra -Iengine $FT_INCS test/rdft_smoke.c engine/rdft.c "$FT_BUILD/libfreetype.a" -o test/rdft_smoke_mipsel
|
||
run_qemu 120 "$QEMU_BIN" ./test/rdft_smoke_mipsel "${FONT:-$ROOT/font/src/MiSans-Regular.otf}"
|
||
|
||
# ---------- 6) qemu-mipsel 书桥冒烟(TXT + 四格式) ----------
|
||
# rdimg.o 引用 stbi_*,实现只在 sdk/meow_ui.a 里(见步骤 3 注释);冒烟不链 UI 主体,
|
||
# 但必须链该归档才能解析 stbi_*(归档按需只拉 stb_image.o)。
|
||
echo "==> mipsel book smoke (TXT)"
|
||
$CC -Os -static -Wall -Wextra -Iapp -Icore -Iengine $FT_INCS $COREINC test/book_smoke.c \
|
||
app/book.c core/rdenc.c core/rdlayout.c core/rdtxt.c core/seg.c core/rdchapter.c core/rdxml.c core/rdhtml.c core/rdsb.c core/rdzip.c \
|
||
core/rdsrc.c core/rdepub.c core/rdfb2.c core/rdmobi.c core/rdcbz.c engine/rdft.c \
|
||
"$OBJ"/mz_*.o "$OBJ"/lm_*.o "$OBJ"/rdimg.o ../sdk/meow_ui.a "$SYSROOT/lib/libz.a" "$FT_BUILD/libfreetype.a" -o test/book_smoke_mipsel
|
||
run_qemu 120 "$QEMU_BIN" ./test/book_smoke_mipsel "${FONT:-$ROOT/font/src/MiSans-Regular.otf}"
|
||
|
||
echo "==> mipsel formats smoke"
|
||
$CC -Os -static -Wall -Wextra -Iapp -Icore -Iengine $FT_INCS $COREINC test/formats_smoke.c \
|
||
app/book.c core/rdenc.c core/rdlayout.c core/rdtxt.c core/seg.c core/rdchapter.c core/rdxml.c core/rdhtml.c core/rdsb.c core/rdzip.c \
|
||
core/rdsrc.c core/rdepub.c core/rdfb2.c core/rdmobi.c core/rdcbz.c engine/rdft.c \
|
||
"$OBJ"/mz_*.o "$OBJ"/lm_*.o "$OBJ"/rdimg.o ../sdk/meow_ui.a "$SYSROOT/lib/libz.a" "$FT_BUILD/libfreetype.a" -o test/formats_smoke_mipsel
|
||
run_qemu 150 "$QEMU_BIN" ./test/formats_smoke_mipsel "${FONT:-$ROOT/font/src/MiSans-Regular.otf}" /tmp/rd_samples
|
||
|
||
# ---------- 7) reader 二进制(mipsel 静态) ----------
|
||
echo "==> reader binary"
|
||
CORE_L="core/rdenc.c core/rdlayout.c core/rdtxt.c core/seg.c core/rdchapter.c core/rdxml.c core/rdhtml.c core/rdsb.c core/rdzip.c core/rdsrc.c core/rdepub.c core/rdfb2.c core/rdmobi.c core/rdcbz.c"
|
||
ENGINE_L="engine/rdft.c"
|
||
APP_L="app.c app/shelf.c app/shelf_model.c app/shelf_index.c app/font_resolve.c app/reader.c app/book.c app/progress.c app/settings.c app/reader_conf.c app/reader_ipc.c"
|
||
$CC -Os -static -Wall -Wextra -I../sdk -I../meow_launcherd -I. -Icore -Iapp -Iengine $FT_INCS $COREINC \
|
||
$CORE_L $APP_L $ENGINE_L "$OBJ"/mz_*.o "$OBJ"/lm_*.o "$OBJ"/rdimg.o \
|
||
../sdk/meow_ui.a "$SYSROOT/lib/libz.a" "$FT_BUILD/libfreetype.a" -lpthread -o reader
|
||
ls -la reader
|
||
file reader
|
||
md5sum reader
|
||
echo " 运行(qemu): qemu-mipsel ./reader"
|
||
|
||
# ---------- 8) 真机耗时基准(不进发行包:与 reader 同源,只把 app.c 的 main 换成 bench main) ----------
|
||
# 用法(设备):MEOW_FONT_* 等 env 与 App 一致时执行 ./shelf_bench_ondev_mipsel [书...]
|
||
echo "==> ondev shelf bench (not shipped)"
|
||
APP_BENCH_L="app/shelf.c app/shelf_model.c app/shelf_index.c app/font_resolve.c app/reader.c app/book.c app/progress.c app/settings.c app/reader_conf.c app/reader_ipc.c"
|
||
$CC -Os -static -Wall -Wextra -I../sdk -I../meow_launcherd -I. -Icore -Iapp -Iengine $FT_INCS $COREINC \
|
||
test/shelf_bench_ondev.c $CORE_L $APP_BENCH_L $ENGINE_L "$OBJ"/mz_*.o "$OBJ"/lm_*.o "$OBJ"/rdimg.o \
|
||
../sdk/meow_ui.a "$SYSROOT/lib/libz.a" "$FT_BUILD/libfreetype.a" -lpthread -o test/shelf_bench_ondev_mipsel
|
||
ls -la test/shelf_bench_ondev_mipsel
|
||
echo "build_reader OK"
|