Files
MeowOS-For-R3P2/patch_v2/reader_app/build_reader.sh
T
AskaEth e0dfab95c3 perf(reader): M32 书架索引落盘 — 冷启动只 stat 434 个目录 mtime,跳过 readdir(3.0s→~0.15s)
M29 只解决了「回书架不重扫」;首次进书架仍要全量扫 434 目录(真机 3.0–3.4s,
其中 opendir+readdir 占 96%)。本单把扫描结果 + 每个扫描目录的 mtime 存到
MeowData/os.meow.reader/index/shelf.idx,冷启动逐目录 stat 比对 mtime,命中即
完全跳过 readdir;任何异常一律 fail-open 全量重扫。

先量再选(任务 1):宿主 434 目录/595 本/51 rar 虚拟库(贴真机量级)
- 434 次目录 stat = 0.48ms/pass(1.1µs/次,不含 readdir);
- 读 72KB 索引 = 0.009ms;全量扫描 3.54ms;索引命中 1.29ms(ts_fn=NULL)。
真机按 M28 实测 1079 次 stat=160–170ms(≈150µs/次)外推:434 次 ≈ 65ms,
远低于「可能不可行」的 1s 阈值 → 选「校验全部扫描目录 mtime」(非只 level-1)。
strace 断言:indexonly 载入 0 次 getdents64、恰好 434 次 stat(真的没 readdir)。
生产路径 ≈ 65ms(stat) + <20ms(索引读) + 40–50ms(rdprog×595) + <20ms(build_rows)
≈ 0.13–0.15s(对比 3.0s)。

索引格式 v1(手工小端序列化,不依赖结构体布局):
header(magic/version/header_size/root_len) + root + dir 记录×(rel,mtime_s,
mtime_ns,books_here) + book 记录×(rel,size,fmt) + trailer(ndirs,nbooks,flags,
depth_skipped,crc32,magic)。truncated/depth_skipped 存在 flags/字段里,载入时
如实重建提示条;**不存阅读进度 ts**,载入后每本由 rdprog_load 现读再排序。

失效语义(存每个扫描过的目录,不只 level-1):
- level-2 新增 / level-3 深处新增 / 删书 → 该目录 mtime 变 → 重扫;
- 一级目录改名 → 父目录 mtime 变 + 旧目录 stat ENOENT → 重扫;
- 超深(>6 层)未扫描子树不在索引内,其内部变化不自动失效(本来就不显示),
  点「刷新」更新 depth_skipped 计数;mtime 精度受限(exFAT 10ms/FAT32 2s)的
  同刻度变更也可能漏判,点「刷新」必可纠正。
M29 语义不变:首次进书架=索引或全扫;回书架=不扫;刷新=删索引+全扫(之后原子
写回新索引)。

安全/健壮(宁可慢不可错):
- 写:shelf.idx.tmp 全量写完(尽力 fsync)后 rename;中途被杀只剩 tmp,正式索引
  要么旧要么完整新;rename 覆盖失败先删旧再改名;fflush/写错即放弃,绝不影响书架。
- 读:O_NOFOLLOW|O_NONBLOCK + S_ISREG(FIFO/符号链接直接拒绝不阻塞);CRC32 覆盖
  全部记录;rel 做绝对路径/“..”/内嵌 NUL/长度校验;版本/根路径/计数/逐目录 mtime
  任一不符即重扫;载入崩溃路径全部 goto 清理,不改书架可用性。
- writer_open 失败(卡满/写保护/无 MeowData)→ 扫描不采集目录、照常出书;
  commit 失败(含只读目录 rename 失败)不留正式索引。

测试:test/shelf_index_smoke 154 pass(含四种变更场景失效、截断/字节翻转/垃圾/
缺失/FIFO/符号链接/版本不符/根路径不符、ts 现读、truncated/depth_skipped 重建、
只读目录写失败、空 Books、root 缺失、.tmp 残渣),ASan+UBSan 干净;
test/shelf_index_bench 模拟 434 目录/595 本/51 rar:scan/load 均 595 本、
truncated=0/depth_skipped=0、索引 72,156B。build_reader.sh 全绿 0 warning
(core 6846 / seg 19 / chapter 71 / shelf_group 63 / index 154 / font 14 /
loaders PASS)。shelf_bench_ondev 增加 index_load/scan_save 计时与
MEOW_BENCH_IDX=miss(真机首开命中 vs 未命中对比,待有卡复核)。

交付:app.json 0.1.4/code5 → 0.1.5/code6;reader md5 df4cadcf
1,711,924B;mpk md5 fec7b1bb 814,535B;未签名不推卡。
2026-11-10 18:14:52 +08:00

197 lines
12 KiB
Bash
Executable File
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
#!/bin/bash
# build_reader.sh — os.meow.reader R2 可复现构建:
# vendor: freetype(静态)/miniz/libmobi/stb/zlib 头 → 交叉 mipsel 编 reader 二进制
# + 宿主 core 单测(R1+loaders) + qemu-mipsel 冒烟(rdft/book/formats)。
#
# 产物:
# reader mipsel 静态 ELF(链 meow_ui.a + freetype + miniz + libmobi + zlib)
# test/test_reader_core_host 宿主 core 单测(enc/layout/txt)
# test/shelf_group_smoke_host 宿主书架分组单测(M27/M31 副题)
# test/shelf_index_smoke_host 宿主书架索引单测(M32:命中/失效/fail-open/ts 不缓存)
# test/shelf_index_bench_host 宿主索引耗时基准(M32,只编不跑;bench/indexonly)
# test/font_resolve_smoke_host 宿主字体路径记忆化单测(M31)
# test/font_resolve_bench_host 宿主字体 walk/命中耗时对比(M31,只编不跑)
# test/loaders_smoke_host 宿主四格式 loader 单测
# test/rdft_smoke_mipsel qemu-mipsel FreeType 冒烟
# test/book_smoke_mipsel qemu-mipsel TXT 书桥冒烟
# test/formats_smoke_mipsel qemu-mipsel 四格式书桥冒烟
#
# 约束:只写本目录(patch_v2/reader_app/);不改 sdk/;-Wall -Wextra 0 warning(自有代码)。
# vendor/(第三方源码+构建产物)gitignore,不入仓;build 脚本可复现。
set -e
cd "$(dirname "$0")"
ROOT="$HOME/Project/R3ProII"
CC="$ROOT/.tools/zig-mipsel-cc"
SYSROOT="$ROOT/.tools/mipsel-linux-muslsf-cross/mipsel-linux-muslsf"
HOST_CC="${HOST_CC:-gcc}"
VENDOR="$PWD/vendor"
OBJ="$VENDOR/build"
# qemu-mipsel 运行时冒烟:宿主没装 qemu(如 Termux 无 mipsel user-mode 包)时,
# 仍编译 mipsel 冒烟二进制,只跳过执行;设 QEMU_BIN=<路径> 可指定其它模拟器。
QEMU_BIN="${QEMU_BIN:-qemu-mipsel}"
run_qemu() {
if [ "${SKIP_QEMU:-0}" = 1 ] || ! command -v "$QEMU_BIN" >/dev/null 2>&1; then
shift
echo " SKIP qemu 执行(未找到 $QEMU_BIN):$*"
return 0
fi
timeout "$@"
}
mkdir -p "$OBJ"
# ---------- 1) vendor 拉取/对齐 ----------
FT_VER="2.14.3"; MINIZ_URL="https://codeload.github.com/richgel999/miniz/tar.gz/refs/heads/master"
# libmobi 版本锁定:见 NOTICE「Locked version (commit)」。LGPL-3.0 §4(d) 要求「已分发二进制
# 对应的 libmobi 源码」与公开源码一致可复现,故不用 refs/heads/master(移动分支),改锁到精确
# commit SHA(2024-10-29 的 9062742...,与上级代理 agent-332 复验/NOTICE 声明一致,已验证可拉取)。
LM_URL="https://codeload.github.com/bfabiszewski/libmobi/tar.gz/906274205c11944b628da1c553b255acb1af7c55"
mkdir -p "$VENDOR"
fetch_freetype() {
if [ -n "$FREETYPE_SRC" ] && [ -f "$FREETYPE_SRC/CMakeLists.txt" ]; then cp -r "$FREETYPE_SRC" vendor/freetype-src;
elif [ ! -d "$VENDOR/freetype-src" ] && [ -f /tmp/freetype/CMakeLists.txt ]; then cp -r /tmp/freetype "$VENDOR/freetype-src";
elif [ ! -d "$VENDOR/freetype-src" ]; then
( cd "$VENDOR" && curl -fsSL -o ft.tar.xz "https://download.savannah.gnu.org/releases/freetype/freetype-$FT_VER.tar.xz" \
&& mkdir -p freetype-src && tar xf ft.tar.xz -C freetype-src --strip-components=1 && rm ft.tar.xz )
fi
[ -f "$VENDOR/freetype-src/CMakeLists.txt" ] || { echo "freetype 不可用"; exit 1; }
}
fetch_miniz() {
if [ ! -f "$VENDOR/miniz/miniz.c" ]; then
( cd /tmp && curl -fsSL -o miniz.tgz "$MINIZ_URL" && mkdir -p miniz_m && tar xzf miniz.tgz -C miniz_m --strip-components=1 \
&& cp miniz_m/miniz.c miniz_m/miniz.h miniz_m/miniz_common.h miniz_m/miniz_tdef.c miniz_m/miniz_tdef.h \
miniz_m/miniz_tinfl.c miniz_m/miniz_tinfl.h miniz_m/miniz_zip.c miniz_m/miniz_zip.h "$VENDOR/miniz/" )
fi
[ -f "$VENDOR/miniz/miniz.c" ] || { echo "miniz 不可用"; exit 1; }
}
fetch_libmobi() {
# 若 vendor/libmobi 来自旧版(LM_URL 改 pin 前抓的 master),请先删 vendor/libmobi 再重跑,
# 以确保取到下方锁定的 commit。此处仅在其缺失时拉取。
if [ ! -f "$VENDOR/libmobi/src/mobi.h" ]; then
( cd /tmp && curl -fsSL -o lm.tgz "$LM_URL" && mkdir -p lm_m && tar xzf lm.tgz -C lm_m --strip-components=1 \
&& mkdir -p "$VENDOR/libmobi" && cp -r lm_m/src "$VENDOR/libmobi/src" && cp lm_m/COPYING "$VENDOR/libmobi/COPYING" )
fi
[ -f "$VENDOR/libmobi/src/mobi.h" ] || { echo "libmobi 不可用"; exit 1; }
}
fetch_stb() { [ -f "$VENDOR/stb/stb_image.h" ] || { mkdir -p "$VENDOR/stb"; cp "$ROOT/patch_v2/sdk/stb_image.h" "$VENDOR/stb/stb_image.h"; }; }
fetch_freetype; fetch_miniz; fetch_libmobi; fetch_stb
if [ ! -f "$VENDOR/zlib/zlib.h" ]; then mkdir -p "$VENDOR/zlib"; cp /usr/include/zlib.h /usr/include/zconf.h "$VENDOR/zlib/" 2>/dev/null || true; fi
# ---------- 2) FreeType mipsel 静态构建 ----------
FT_SRC="$VENDOR/freetype-src"; FT_BUILD="$VENDOR/freetype-build"
if [ ! -f "$FT_BUILD/libfreetype.a" ] || [ -n "$FORCE_FT" ]; then
rm -rf "$FT_BUILD"
cmake -S "$FT_SRC" -B "$FT_BUILD" -DCMAKE_SYSTEM_NAME=Linux -DCMAKE_C_COMPILER="$CC" \
-DCMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF \
-DFT_DISABLE_ZLIB=TRUE -DFT_DISABLE_BZIP2=TRUE -DFT_DISABLE_PNG=TRUE -DFT_DISABLE_HARFBUZZ=TRUE -DFT_DISABLE_BROTLI=TRUE >/dev/null
cmake --build "$FT_BUILD" -j"$(nproc)" >/dev/null 2>&1
fi
FT_INCS="-I$FT_BUILD -I$FT_BUILD/include -I$FT_SRC/include"
COREINC="-Icore -Ivendor/miniz -Ivendor/stb -Ivendor/libmobi/src -Ivendor/zlib"
MZFLAGS="-DMINIZ_NO_ZLIB_COMPATIBLE_NAMES -DMINIZ_NO_TIME"
# ---------- 3) miniz + libmobi 对象(mipsel) ----------
build_mipsel_libs() {
for f in vendor/miniz/miniz_tinfl.c vendor/miniz/miniz_tdef.c vendor/miniz/miniz_zip.c vendor/miniz/miniz.c; do
b="$(basename "$f" .c)"; [ -f "$OBJ/mz_$b.o" ] || $CC -Os -static -Wall -Wextra -Ivendor/miniz $MZFLAGS -c "$f" -o "$OBJ/mz_$b.o"
done
for f in vendor/libmobi/src/*.c; do b="$(basename "$f" .c)"; [ "$b" = "miniz" ] && continue
[ -f "$OBJ/lm_$b.o" ] || $CC -Os -static -w -Ivendor/libmobi/src -Ivendor/zlib -c "$f" -o "$OBJ/lm_$b.o"
done
# stb_image 的实现由 meow_ui.a 提供(SDK 自 M6.45 收编 stb_image.c.o);此处只编封装,
# 否则链接 meow_ui.a 时 stbi_* 会与归档内 stb_image.o 重复定义。
[ -f "$OBJ/rdimg.o" ] || $CC -Os -static -Wall -Wextra -Ivendor/stb -c engine/rdimg.c -o "$OBJ/rdimg.o"
}
build_mipsel_libs
# ---------- 4) 宿主 loader 单测(miniz/libmobi 用宿主编一遍) ----------
echo "==> host core test (R1)"
$HOST_CC -O2 -Wall -Wextra -Icore -I. test/test_reader_core.c core/rdenc.c core/rdlayout.c core/rdtxt.c -o test/test_reader_core_host -lm
./test/test_reader_core_host
echo "==> host seg test (R3.2)"
$HOST_CC -O2 -Wall -Wextra -Icore test/seg_smoke.c core/seg.c core/rdenc.c -o test/seg_smoke_host -lm
./test/seg_smoke_host
echo "==> host chapter test (R3.3)"
$HOST_CC -O2 -Wall -Wextra -Icore test/chapter_smoke.c core/rdchapter.c core/seg.c core/rdenc.c -o test/chapter_smoke_host -lm
./test/chapter_smoke_host
echo "==> host shelf group test (M27)"
$HOST_CC -O2 -Wall -Wextra -Iapp -I../sdk test/shelf_group_smoke.c app/shelf_model.c app/progress.c app/reader_conf.c -o test/shelf_group_smoke_host
./test/shelf_group_smoke_host
echo "==> host shelf index test (M32)"
$HOST_CC -O2 -Wall -Wextra -Iapp -I../sdk test/shelf_index_smoke.c app/shelf_model.c app/shelf_index.c app/progress.c -o test/shelf_index_smoke_host
./test/shelf_index_smoke_host
$HOST_CC -O2 -Wall -Wextra -Iapp -I../sdk test/shelf_index_bench.c app/shelf_model.c app/shelf_index.c -o test/shelf_index_bench_host
echo "==> host font resolve test (M31)"
$HOST_CC -O2 -Wall -Wextra -Iapp test/font_resolve_smoke.c app/font_resolve.c -o test/font_resolve_smoke_host
./test/font_resolve_smoke_host
$HOST_CC -O2 -Wall -Wextra -Iapp test/font_resolve_bench.c app/font_resolve.c -o test/font_resolve_bench_host
echo "==> host shelf scan bench (M27,只编不跑;用法 ./test/shelf_scan_bench_host <Books目录> [progress|noop] [轮数])"
$HOST_CC -O2 -Wall -Wextra -Iapp -I../sdk test/shelf_scan_bench.c app/shelf_model.c app/progress.c -o test/shelf_scan_bench_host
echo "==> host loaders test"
HOBJ="$OBJ/host"; mkdir -p "$HOBJ"
for f in vendor/miniz/miniz_tinfl.c vendor/miniz/miniz_tdef.c vendor/miniz/miniz_zip.c vendor/miniz/miniz.c; do
b="$(basename "$f" .c)"; [ -f "$HOBJ/mz_$b.o" ] || $HOST_CC -O2 -Ivendor/miniz $MZFLAGS -c "$f" -o "$HOBJ/mz_$b.o"
done
for f in vendor/libmobi/src/*.c; do b="$(basename "$f" .c)"; [ "$b" = "miniz" ] && continue
[ -f "$HOBJ/lm_$b.o" ] || $HOST_CC -O2 -w -Ivendor/libmobi/src -Ivendor/zlib -c "$f" -o "$HOBJ/lm_$b.o"
done
HCORE=""
for f in rdxml rdhtml rdsb rdzip rdsrc rdepub rdfb2 rdmobi rdcbz rdenc rdtxt; do
$HOST_CC -O2 -Wall -Wextra $COREINC -c core/$f.c -o "$HOBJ/core_$f.o"
done
python3 tools/gen_samples.py >/dev/null
$HOST_CC -O2 -Wall -Wextra $COREINC test/loaders_smoke.c "$HOBJ"/*.o -lz -lm -o test/loaders_smoke_host
./test/loaders_smoke_host /tmp/rd_samples
# ---------- 5) qemu-mipsel FreeType 冒烟 ----------
echo "==> mipsel ft smoke"
$CC -Os -static -Wall -Wextra -Iengine $FT_INCS test/rdft_smoke.c engine/rdft.c "$FT_BUILD/libfreetype.a" -o test/rdft_smoke_mipsel
run_qemu 120 "$QEMU_BIN" ./test/rdft_smoke_mipsel "${FONT:-$ROOT/font/src/MiSans-Regular.otf}"
# ---------- 6) qemu-mipsel 书桥冒烟(TXT + 四格式) ----------
# rdimg.o 引用 stbi_*,实现只在 sdk/meow_ui.a 里(见步骤 3 注释);冒烟不链 UI 主体,
# 但必须链该归档才能解析 stbi_*(归档按需只拉 stb_image.o)。
echo "==> mipsel book smoke (TXT)"
$CC -Os -static -Wall -Wextra -Iapp -Icore -Iengine $FT_INCS $COREINC test/book_smoke.c \
app/book.c core/rdenc.c core/rdlayout.c core/rdtxt.c core/seg.c core/rdchapter.c core/rdxml.c core/rdhtml.c core/rdsb.c core/rdzip.c \
core/rdsrc.c core/rdepub.c core/rdfb2.c core/rdmobi.c core/rdcbz.c engine/rdft.c \
"$OBJ"/mz_*.o "$OBJ"/lm_*.o "$OBJ"/rdimg.o ../sdk/meow_ui.a "$SYSROOT/lib/libz.a" "$FT_BUILD/libfreetype.a" -o test/book_smoke_mipsel
run_qemu 120 "$QEMU_BIN" ./test/book_smoke_mipsel "${FONT:-$ROOT/font/src/MiSans-Regular.otf}"
echo "==> mipsel formats smoke"
$CC -Os -static -Wall -Wextra -Iapp -Icore -Iengine $FT_INCS $COREINC test/formats_smoke.c \
app/book.c core/rdenc.c core/rdlayout.c core/rdtxt.c core/seg.c core/rdchapter.c core/rdxml.c core/rdhtml.c core/rdsb.c core/rdzip.c \
core/rdsrc.c core/rdepub.c core/rdfb2.c core/rdmobi.c core/rdcbz.c engine/rdft.c \
"$OBJ"/mz_*.o "$OBJ"/lm_*.o "$OBJ"/rdimg.o ../sdk/meow_ui.a "$SYSROOT/lib/libz.a" "$FT_BUILD/libfreetype.a" -o test/formats_smoke_mipsel
run_qemu 150 "$QEMU_BIN" ./test/formats_smoke_mipsel "${FONT:-$ROOT/font/src/MiSans-Regular.otf}" /tmp/rd_samples
# ---------- 7) reader 二进制(mipsel 静态) ----------
echo "==> reader binary"
CORE_L="core/rdenc.c core/rdlayout.c core/rdtxt.c core/seg.c core/rdchapter.c core/rdxml.c core/rdhtml.c core/rdsb.c core/rdzip.c core/rdsrc.c core/rdepub.c core/rdfb2.c core/rdmobi.c core/rdcbz.c"
ENGINE_L="engine/rdft.c"
APP_L="app.c app/shelf.c app/shelf_model.c app/shelf_index.c app/font_resolve.c app/reader.c app/book.c app/progress.c app/settings.c app/reader_conf.c app/reader_ipc.c"
$CC -Os -static -Wall -Wextra -I../sdk -I../meow_launcherd -I. -Icore -Iapp -Iengine $FT_INCS $COREINC \
$CORE_L $APP_L $ENGINE_L "$OBJ"/mz_*.o "$OBJ"/lm_*.o "$OBJ"/rdimg.o \
../sdk/meow_ui.a "$SYSROOT/lib/libz.a" "$FT_BUILD/libfreetype.a" -lpthread -o reader
ls -la reader
file reader
md5sum reader
echo " 运行(qemu): qemu-mipsel ./reader"
# ---------- 8) 真机耗时基准(不进发行包:与 reader 同源,只把 app.c 的 main 换成 bench main) ----------
# 用法(设备):MEOW_FONT_* 等 env 与 App 一致时执行 ./shelf_bench_ondev_mipsel [书...]
echo "==> ondev shelf bench (not shipped)"
APP_BENCH_L="app/shelf.c app/shelf_model.c app/shelf_index.c app/font_resolve.c app/reader.c app/book.c app/progress.c app/settings.c app/reader_conf.c app/reader_ipc.c"
$CC -Os -static -Wall -Wextra -I../sdk -I../meow_launcherd -I. -Icore -Iapp -Iengine $FT_INCS $COREINC \
test/shelf_bench_ondev.c $CORE_L $APP_BENCH_L $ENGINE_L "$OBJ"/mz_*.o "$OBJ"/lm_*.o "$OBJ"/rdimg.o \
../sdk/meow_ui.a "$SYSROOT/lib/libz.a" "$FT_BUILD/libfreetype.a" -lpthread -o test/shelf_bench_ondev_mipsel
ls -la test/shelf_bench_ondev_mipsel
echo "build_reader OK"