rust-webassemblyとコマンドラインでpulldown-cmarkを使って整形してみた
例
その1 wasm
README.ja.mdを読み込み、pulldown_cmarkのwasmでHTMLに整形して、スクリプトで表示位置に流し込んでます。
wasm-pack build --target web
での例。
Manjaro-JP
wasm-pack build --target no-modules
での例。
Manjaro-JP
その2 wasm pulldown_cmark editor
Textareaの変更に伴い、整形して表示します。
Textareaの初期値にREADME.ja.mdを読み込ませています。
wasm-pack build --target web
での例。
Manjaro-JP
wasm-pack build --target no-modules
での例。
Manjaro-JP
その3 コマンドラインで整形したファイルを読み込む形で表示しています。
README.ja.mdをコマンドラインで加工して、README.ja.htmlとして保存。
objectタグでの表示が一部ブラウザで動作しないので、スクリプトのfetchコマンドで読み込んで、表示位置に流し込むように変更しました。
Manjaro-JP
wasm
とりあえず、プロジェクトを作る
cargo new --lib wasm-markdown
cd wasm-markdown
cargo add pulldown-cmark@0.9.3
cargo add wasm-bindgen
Cargo.tomlの中身
[package]
name = "wasm-markdown"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
pulldown-cmark = "0.9.3"
wasm-bindgen = "0.2.87"
[lib]
crate-type = ["cdylib", "rlib"]
[package.metadata.wasm-pack.profile.release]
wasm-opt = ["-Oz", "--enable-mutable-globals"]
[package.metadata.wasm-pack.profile.dev]
wasm-opt = ["-Oz", "--enable-mutable-globals"]
src/lib.rsの中身
use pulldown_cmark::{html, Options, Parser};
use wasm_bindgen::prelude::*;
#[wasm_bindgen]
pub fn pulldown_cmark(source_text: &str) -> String {
// Set up options and parser. Strikethroughs are not part of the CommonMark standard
// and we therefore must enable it explicitly.
let mut options = Options::empty();
options.insert(Options::ENABLE_TABLES);
options.insert(Options::ENABLE_FOOTNOTES);
options.insert(Options::ENABLE_STRIKETHROUGH);
options.insert(Options::ENABLE_TASKLISTS);
options.insert(Options::ENABLE_SMART_PUNCTUATION);
options.insert(Options::ENABLE_HEADING_ATTRIBUTES);
let parser = Parser::new_ext(source_text, options);
// Write to String buffer.
let mut html_output = String::new();
html::push_html(&mut html_output, parser);
html_output
}
wasm-packのインストール
curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
または
cargo install wasm-pack
export PATH+=":$HOME/.cargo/bin"
ビルドします。
pkgフォルダに出来ます。
cargo check
cargo build
wasm-pack build --target web
ls -l pkg
rust nightlyを利用する場合の一例
alias wasm-pack-nightly='rustup run nightly wasm-pack "$@"'
wasm-pack-nightly build --target web
wasm その2 git版
cargo add pulldown-cmark --git https://github.com/raphlinus/pulldown-cmark.git --branch master
Cargo.tomlの中身
[package]
name = "wasm-markdown"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
pulldown-cmark = { git = "https://github.com/raphlinus/pulldown-cmark.git", branch = "master", version = "0.9.2" }
wasm-bindgen = "0.2.87"
[lib]
crate-type = ["cdylib", "rlib"]
[package.metadata.wasm-pack.profile.release]
wasm-opt = ["-Oz", "--enable-mutable-globals"]
[package.metadata.wasm-pack.profile.dev]
wasm-opt = ["-Oz", "--enable-mutable-globals"]
src/lib.rsの中身
use pulldown_cmark::{html, Options, Parser};
use wasm_bindgen::prelude::*;
#[wasm_bindgen]
pub fn pulldown_cmark(text: &str) -> String {
let mut opts = Options::empty();
opts.insert(Options::ENABLE_TABLES);
opts.insert(Options::ENABLE_FOOTNOTES);
opts.insert(Options::ENABLE_OLD_FOOTNOTES);
opts.insert(Options::ENABLE_STRIKETHROUGH);
opts.insert(Options::ENABLE_TASKLISTS);
opts.insert(Options::ENABLE_SMART_PUNCTUATION);
opts.insert(Options::ENABLE_HEADING_ATTRIBUTES);
opts.insert(Options::ENABLE_YAML_STYLE_METADATA_BLOCKS);
opts.insert(Options::ENABLE_PLUSES_DELIMITED_METADATA_BLOCKS);
let p = Parser::new_ext(text, opts);
let mut html_output = String::new();
html::push_html(&mut html_output, p);
html_output
}
ビルドします。
pkgフォルダに出来ます。
cargo update
cargo check
cargo build
wasm-pack build --target web
ls -l pkg
JavaScriptからの呼び出し例
fetchでMarkdownファイルを読み込んで、HTMLに整形させる例。
createTextNodeに取り込んでいるのは、html表示でescapeが必要な文字の対策のためです。
highlight.jsも利用。
<link rel="stylesheet" href="./sunburst.min.css">
<script src="./highlight.min.js"></script>
<script type="module">
import init, { pulldown_cmark as wasm } from './wasm-markdown/wasm_markdown.js';
var markdown_file = './README.ja.md';
window.onload=function() {
init()
.then(() => {
fetch(markdown_file)
.then(response => response.text())
.then(data => {
let el = document.getElementById( "insertText" );
let text = document.createTextNode(wasm(data));
el.innerHTML = text.data;
hljs.highlightAll();
addCopybtn();
});
});
function addCopybtn(){
Array.prototype.forEach.call(document.querySelectorAll('pre > code'), addSelectButton);
}
function addSelectButton(elmCodeFrame){
var elmSelectButton = document.createElement('div');
elmSelectButton.setAttribute('class', '__select-code');
elmSelectButton.setAttribute('style', "\
display: inline-block;\
left: 90%;\
float: right;\
font-weight: bold;\
background-color: transparent;\
cursor: pointer;")
elmSelectButton.addEventListener('click', function(){
select(elmCodeFrame);
});
elmSelectButton.insertAdjacentHTML('afterbegin', '<span>Copy</span><i class="fa fa-clipboard"/>');
elmCodeFrame.insertBefore(elmSelectButton, elmCodeFrame.firstChild);
console.log(elmSelectButton);
}
function select(elm){
var selection = window.getSelection();
selection.removeAllRanges();
var range = document.createRange();
range.selectNodeContents(elm);
selection.addRange(range);
document.execCommand('copy');
selection.removeAllRanges();
var span = elm.querySelector('.__select-code span');
span.innerHTML="Copied";
span.style.transition = 'none';
setTimeout(function() {
span.style.opacity = 0;
span.style.transition = 'opacity 1s';
}, 500);
setTimeout(function() {
span.innerHTML="Copy";
span.style.opacity =1;
}, 1000);
}
};
</script>
</head>
<body>
<header></header>
<div id="insertText"></div>
<footer></footer>
コマンドライン
ファイルまたは、標準入力からのデータを変換して、標準出力に結果を出力する、簡単なコマンドを用意してみる。git版の最新ソースをベースにファイルからの読み込み機能を追加します。
cargo new pulldown-cmark
cd pulldown-cmark
cargo add pulldown-cmark --git https://github.com/raphlinus/pulldown-cmark.git --branch master
cargo add getopts
Cargo.tomlの中身
[package]
name = "pulldown-cmark"
version = "0.9.2"
authors = ["Masato TOYOSHIMA <phoepsilonix@phoepsilonix.love>", "Raph Levien <raph.levien@gmail.com>", "Marcus Klaas de Vries <mail@marcusklaas.nl>" ]
edition = "2021"
description = "Reads markdown from file or standard input and emits HTML."
repository = "https://github.com/raphlinus/pulldown-cmark"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
getopts = "0.2.21"
pulldown-cmark = { git = "https://github.com/raphlinus/pulldown-cmark.git", branch = "master", version = "0.9.2" }
src/main.rsの中身
// Copyright 2015 Google Inc. All rights reserved.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
//! Command line tool to exercise pulldown-cmark.
#![forbid(unsafe_code)]
use pulldown_cmark::{html, Options, Parser};
use std::env;
use std::io::{self, Read};
use std::mem;
use std::fs::File;
use std::path::PathBuf;
fn dry_run(text: &str, opts: Options) {
let p = Parser::new_ext(text, opts);
let count = p.count();
println!("{} events", count);
}
fn print_events(text: &str, opts: Options) {
let parser = Parser::new_ext(text, opts).into_offset_iter();
for (event, range) in parser {
println!("{:?}: {:?}", range, event);
}
println!("EOF");
}
fn brief(program: &str) -> String {
format!(
"Usage: {} [options]\n\n{}",
program, "Reads markdown from file or standard input and emits HTML.",
)
}
pub fn main() -> std::io::Result<()> {
let args: Vec<_> = env::args().collect();
let mut opts = getopts::Options::new();
opts.optflag("h", "help", "this help message");
opts.optflag("d", "dry-run", "dry run, produce no output");
opts.optflag("e", "events", "print event sequence instead of rendering");
opts.optflag("T", "enable-tables", "enable GitHub-style tables");
opts.optflag("F", "enable-footnotes", "enable GitHub-style footnotes");
opts.optflag("", "enable-old-footnotes", "enable Hoedown-style footnotes");
opts.optflag(
"S",
"enable-strikethrough",
"enable GitHub-style strikethrough",
);
opts.optflag("L", "enable-tasklists", "enable GitHub-style task lists");
opts.optflag("P", "enable-smart-punctuation", "enable smart punctuation");
opts.optflag(
"H",
"enable-heading-attributes",
"enable heading attributes",
);
opts.optflag("M", "enable-metadata-blocks", "enable metadata blocks");
let matches = match opts.parse(&args[1..]) {
Ok(m) => m,
Err(f) => {
eprintln!("{}\n{}", f, opts.usage(&brief(&args[0])));
std::process::exit(1);
}
};
if matches.opt_present("help") {
println!("{}", opts.usage(&brief(&args[0])));
return Ok(());
}
let mut opts = Options::empty();
if matches.opt_present("enable-tables") {
opts.insert(Options::ENABLE_TABLES);
}
if matches.opt_present("enable-footnotes") {
opts.insert(Options::ENABLE_FOOTNOTES);
}
if matches.opt_present("enable-old-footnotes") {
opts.insert(Options::ENABLE_OLD_FOOTNOTES);
}
if matches.opt_present("enable-strikethrough") {
opts.insert(Options::ENABLE_STRIKETHROUGH);
}
if matches.opt_present("enable-tasklists") {
opts.insert(Options::ENABLE_TASKLISTS);
}
if matches.opt_present("enable-smart-punctuation") {
opts.insert(Options::ENABLE_SMART_PUNCTUATION);
}
if matches.opt_present("enable-heading-attributes") {
opts.insert(Options::ENABLE_HEADING_ATTRIBUTES);
}
if matches.opt_present("enable-metadata-blocks") {
opts.insert(Options::ENABLE_YAML_STYLE_METADATA_BLOCKS);
opts.insert(Options::ENABLE_PLUSES_DELIMITED_METADATA_BLOCKS);
}
let mut input = String::new();
if !&matches.free.is_empty() {
for filename in &matches.free {
let real_path = PathBuf::from(filename);
let mut f = File::open(&real_path).expect("file not found");
f.read_to_string(&mut input)
.expect("something went wrong reading the file");
if matches.opt_present("events") {
print_events(&input, opts);
} else if matches.opt_present("dry-run") {
dry_run(&input, opts);
} else {
pulldown_cmark(&input, opts);
}
}
} else {
let _ = io::stdin().lock().read_to_string(&mut input);
if matches.opt_present("events") {
print_events(&input, opts);
} else if matches.opt_present("dry-run") {
dry_run(&input, opts);
} else {
pulldown_cmark(&input, opts);
}
}
Ok(())
}
pub fn pulldown_cmark(input: &str, opts: Options) {
let mut p = Parser::new_ext(input, opts);
let mut buffer = String::new();
html::push_html(&mut buffer, &mut p);
print!("{}", buffer);
/*
let stdio = io::stdout();
let buffer = std::io::BufWriter::with_capacity(1024 * 1024, stdio.lock());
let _ = html::write_html(buffer, &mut p);
*/
// Since the program will now terminate and the memory will be returned
// to the operating system anyway, there is no point in tidely cleaning
// up all the datastructures we have used. We shouldn't do this if we'd
// do other things after this, because this is basically intentionally
// leaking data. Skipping cleanup lets us return a bit (~5%) faster.
mem::forget(p);
}
ビルド
stripもしてみます。
cargo check
cargo build --release
ls -l target/release/pulldown-cmark
-rwxr-xr-x 2 phoepsilonix phoepsilonix 5532984 10月 14 17:04 target/release/pulldown-cmark
strip --strip-all target/release/pulldown-cmark
ls -l target/release/pulldown-cmark
または、
cargo check
RUSTFLAGS="-C strip=symbols" cargo build --release
ls -l target/release/pulldown-cmark
-rwxr-xr-x 2 phoepsilonix phoepsilonix 837992 10月 14 17:05 target/release/pulldown-cmark
sudo cp target/release/pulldown-cmark /usr/local/bin
実行例
/usr/local/bin/pulldown-cmark -h
Usage: pulldown-cmark [options]
Reads markdown from file or standard input and emits HTML.
Options:
-h, --help this help message
-d, --dry-run dry run, produce no output
-e, --events print event sequence instead of rendering
-T, --enable-tables
enable GitHub-style tables
-F, --enable-footnotes
enable GitHub-style footnotes
--enable-old-footnotes
enable Hoedown-style footnotes
-S, --enable-strikethrough
enable GitHub-style strikethrough
-L, --enable-tasklists
enable GitHub-style task lists
-P, --enable-smart-punctuation
enable smart punctuation
-H, --enable-heading-attributes
enable heading attributes
-M, --enable-metadata-blocks
enable metadata blocks
/usr/local/bin/pulldown-cmark -TFSLPHM input_1.md > output.html
/usr/local/bin/pulldown-cmark -TFSLPHM input_1.md input_2.md > output.html
cat input*.md | /usr/local/bin/pulldown-cmark -TFSLPHM > output.html
コマンドライン その2 clapでコマンドラインのオプション処理をしてみる。
コマンドラインのオプション処理にclapを用いて、書き換えてみた。
ファイルサイズが、若干、大きくなりました。
cargo new pulldown-cmark
cd pulldown-cmark
cargo add pulldown-cmark --git https://github.com/raphlinus/pulldown-cmark.git --branch master
cargo add clap --features derive
Cargo.tomlの中身
[package]
name = "pulldown-cmark"
version = "0.9.2"
edition = "2021"
description = "Reads markdown from file or standard input and emits HTML."
authors = ["Masato TOYOSHIMA <phoepsilonix@phoepsilonix.love>", "Raph Levien <raph.levien@gmail.com>", "Marcus Klaas de Vries <mail@marcusklaas.nl>" ]
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
clap = { version = "4.4.6", features = ["derive"] }
pulldown-cmark = { git = "https://github.com/raphlinus/pulldown-cmark.git", branch = "master", version = "0.9.2" }
src/main.rsの中身
use std::env;
use std::fs::File;
use std::path::PathBuf;
use std::io::{self, Read};
use std::mem;
use pulldown_cmark::{html, Options, Parser as mdParser};
use clap::Parser;
#[derive(Parser,Debug)]
#[clap(
name = env!("CARGO_PKG_NAME"),
version = env!("CARGO_PKG_VERSION"),
author = env!("CARGO_PKG_AUTHORS"),
about = env!("CARGO_PKG_DESCRIPTION"),
arg_required_else_help = false)]
#[allow(non_snake_case)]
struct AppArg {
#[arg(short('T'), long = "enable-tables", long_help = "enable Github-style tables")]
T: bool,
#[arg(short('F'), long = "enable-footnotes", long_help = "enable Github-style footnotes")]
F: bool,
#[arg(short('S'), long = "enable-strikethrough", long_help = "enable GitHub-style strikethrough")]
S: bool,
#[arg(short('L'), long = "enable-tasklists", long_help = "enable GitHub-style task lists")]
L: bool,
#[arg(short('P'), long = "enable-smart-punctuation", long_help = "enable smart punctuation")]
P: bool,
#[arg(short('H'), long = "enable-heading-attributes", long_help = "enable heading attributes")]
H: bool,
#[arg(short('M'), long = "enable-metadata-blocks", long_help = "enable metadata blocks")]
M: bool,
#[arg(short('d'), long = "dry-run", long_help = "dry run, produce no output")]
d: bool,
#[arg(short('e'), long = "events", long_help = "print event sequence instead of rendering")]
e: bool,
#[arg(help = "filename")]
filename: Vec<String>,
}
fn dry_run(text: &str, opts: Options) {
let p = mdParser::new_ext(text, opts);
let count = p.count();
println!("{} events", count);
}
fn print_events(text: &str, opts: Options) {
let parser = mdParser::new_ext(text, opts).into_offset_iter();
for (event, range) in parser {
println!("{:?}: {:?}", range, event);
}
println!("EOF");
}
fn main() {
let arg: AppArg = AppArg::parse();
let mut opts = Options::empty();
if arg.T { opts.insert(Options::ENABLE_TABLES); }
if arg.F {
opts.insert(Options::ENABLE_FOOTNOTES);
opts.insert(Options::ENABLE_OLD_FOOTNOTES);
}
if arg.S { opts.insert(Options::ENABLE_STRIKETHROUGH); }
if arg.L { opts.insert(Options::ENABLE_TASKLISTS); }
if arg.P { opts.insert(Options::ENABLE_SMART_PUNCTUATION); }
if arg.H { opts.insert(Options::ENABLE_HEADING_ATTRIBUTES); }
if arg.M {
opts.insert(Options::ENABLE_YAML_STYLE_METADATA_BLOCKS);
opts.insert(Options::ENABLE_PLUSES_DELIMITED_METADATA_BLOCKS);
}
let mut input = String::new();
if !arg.filename.is_empty() {
for filename in arg.filename {
let real_path = PathBuf::from(filename);
let mut f = File::open(&real_path).expect("file not found");
f.read_to_string(&mut input)
.expect("something went wrong reading the file");
if arg.e {
print_events(&input, opts);
} else if arg.d {
dry_run(&input, opts);
} else {
pulldown_cmark(&input, opts);
}
}
} else {
let _ = io::stdin().lock().read_to_string(&mut input);
if arg.e {
print_events(&input, opts);
} else if arg.d {
dry_run(&input, opts);
} else {
pulldown_cmark(&input, opts);
}
}
}
pub fn pulldown_cmark(text: &str, opts: Options) {
let mut p = mdParser::new_ext(text, opts);
let mut buffer = String::new();
html::push_html(&mut buffer, &mut p);
print!("{}", buffer);
mem::forget(p);
}
ビルド
stripもしてみます。
cargo check
cargo build --release
ls -l target/release/pulldown-cmark
-rwxr-xr-x 2 phoepsilonix phoepsilonix 6119272 10月 14 17:18 target/release/pulldown-cmark
strip --strip-all target/release/pulldown-cmark
ls -l target/release/pulldown-cmark
または、
cargo check
RUSTFLAGS="-C strip=symbols" cargo build --release
ls -l target/release/pulldown-cmark
-rwxr-xr-x 2 phoepsilonix phoepsilonix 1280360 10月 14 17:19 target/release/pulldown-cmark
sudo cp target/release/pulldown-cmark /usr/local/bin
実行例
/usr/local/bin/pulldown-cmark -h
Reads markdown from file or standard input and emits HTML.
Usage: pulldown-cmark [OPTIONS] [FILENAME]...
Arguments:
[FILENAME]... filename
Options:
-T, --enable-tables enable Github-style tables
-F, --enable-footnotes enable Github-style footnotes
-S, --enable-strikethrough enable GitHub-style strikethrough
-L, --enable-tasklists enable GitHub-style task lists
-P, --enable-smart-punctuation enable smart punctuation
-H, --enable-heading-attributes enable heading attributes
-M, --enable-metadata-blocks enable metadata blocks
-d, --dry-run dry run, produce no output
-e, --events print event sequence instead of rendering
-h, --help Print help (see more with '--help')
-V, --version Print version
/usr/local/bin/pulldown-cmark -TFSLPHM input_1.md > output.html
/usr/local/bin/pulldown-cmark -TFSLPHM input_1.md input_2.md > output.html
cat input*.md | /usr/local/bin/pulldown-cmark -TFSLPH > output.html
コマンドライン その3 crateに添付されているコマンドを、そのまま利用する
rustのcrateで提供されているコマンドをそのままビルドして用いる場合。
aria2c -c https://static.crates.io/crates/pulldown-cmark/pulldown-cmark-0.9.3.crate -o pulldown-cmark-0.9.3.tar.gz
tar xf pulldown-cmark-0.9.3.tar.gz
cd pulldown-cmark-0.9.3
cargo test --frozen
RUSTFLAGS="-C strip=symbols" cargo build --frozen --release
ls -l target/release/pulldown-cmark
-rwxr-xr-x 2 phoepsilonix phoepsilonix 825664 9月 28 05:21 target/release/pulldown-cmark
target/release/pulldown-cmark -h
Usage: target/release/pulldown-cmark [options]
Reads markdown from standard input and emits HTML.
Options:
-h, --help this help message
-d, --dry-run dry run, produce no output
-e, --events print event sequence instead of rendering
-T, --enable-tables
enable GitHub-style tables
-F, --enable-footnotes
enable Hoedown-style footnotes
-S, --enable-strikethrough
enable GitHub-style strikethrough
-L, --enable-tasklists
enable GitHub-style task lists
-P, --enable-smart-punctuation
enable smart punctuation
-H, --enable-heading-attributes
enable heading attributes
標準入力からのデータを変換してくれます。
実行例
pulldown-cmark -TFSLPH < input_filename.md > output_filename.html
cat input_filename.md | pulldown-cmark -TFSLPH > output_filename.html
コマンドライン その4 aur
Arch Linux のaurパッケージを利用する場合。
paru -S pulldown-cmark
または
yay -S pulldown-cmark
実行例
pulldown-cmark -TFSLPH < input_filename.md > output_filename.html
cat input_filename.md | pulldown-cmark -TFSLPH > output_filename.html
git版 v0.9.3-dev バイナリ
cargo zigbuildでクロスコンパイルしたバイナリを用意しました。
Releases · phoepsilonix/pulldown-cmark
参考サイト
Rust と WebAssembly で爆速な Markdown Editor を作ってみる
Qiita のコードの右上にコピーボタンを追加するユーザースクリプト - Qiita
highlight.js
シンタックス・ハイライト・ライブラリ「highlight.js」の使い方 – ラボラジアン
Discussion