⏱️
UnityのキャッシュサーバAcceleratorをコンテナで動かす
Unity Accelerator
開発速度を上げるキャッシュサーバなのになんかあんまり知名度がない気がするUnity Acceleratorです。
これをローカルマシン上にコンテナで立ち上げて、それをUnityから使うのがこの記事の説明です。
環境
Unity Accelerator v1.0.946+g7cc8240
コンテナで立ち上げ
ローカルマシンにインストールする方法とコンテナで立ち上げる方法が提示されていますが、コンテナがいいと思います。理由は以下です。
- ダッシュボードを見るためのポート10080がブラウザ標準でブロックされている
- いろんなディレクトリが汚染される
- ローカルにインストールした場合、終了するのがすごく面倒
特にひとつめの10080が大きいですね。Acceleratorが使用するポートを環境変数などで指定できないので、簡単に変更できません。なのでコンテナにポートフォワーディングしてもらって簡単にダッシュボードを見よう、がこの章の目的です。
composeで立ち上げる
これをdocker compose upやnerdctl compose upで起動してください。おわり。
なんかいろいろ動いてるのを見たいのと、簡単に終了したいので-dはないほうがいいと思います。
ports
ports:
- "5678:10080"
ポートフォワーディングしているのはここです。10080を誰も使ってなさそうな5678にしています。変えたい場合はここを変えてください。この記事では5678にしたものとして話を進めます。
動かす
無事に立ち上がったらいろいろ動かしてみます。
ダッシュボード
Metrics
まずダッシュボードを開くとMetricsです。
説明が……ない……！
Internet Data Downloaded into cacheがキャッシュされているアセットのサイズなのかな……？ という推測ができますが、後でエディタと繋いで動きを見るのでいったん忘れてください。
ログイン
ConfigurationとLogsにはログインを要求されるので、docker-composeで設定したadmin/adminを入力します。
ここも変えたい場合はなんか好きに変えてください。
user/pass
environment:
- USER=admin
- PASSWORD=admin
Configuration
説明が……ない……！
とはいえこちらは設定なのでまあわからなくはないです。
なんでCurrent Asset Import Pipeline V2 Portのportは変更できないんでしょうか。今使ってるからってこと？ なぞ。
基本的に触らなくていいと思いますが、いじるなら下にあるData Policyでしょうか？ 使いすぎたらいい感じに開放してくれるそうです。
もしくは、自動でAccelerator本体をバージョンアップするらしいので、MaintenanceのAuto Updateをオフにしておくなど。
Logs
説明が……いらない……！
jsonで構造化されたログを出しています。
AWS上に立てる場合なんかは費用計算で必要になると思います。
UnityEditor
接続
UnityEditorにAcceleratorを接続するには以下3つの方法があります。
-
Edit > PreferencesでUnityEditorそのものに設定 -
ProjectSettings > Editor > Cache Serverでプロジェクト毎に設定 - CLIの引数から設定
今回は2つめのプロジェクト毎に設定する方法を使います。
ModeをEnabledに設定し、localhost:5678と入力してCheck Connectionを押します。
Success! したらOKです。このときcomposeを立ち上げたウインドウもログが動いていたら更に良し。
ここで見えている設定はEditorSettings.cache~で変更できます。エディタ拡張などで環境によって変更できるようにしておくと便利だと思います。
// 接続先を変える
EditorSettings.cacheServerEndpoint = "localhost:5678";
画面の右下のアイコンでも接続状態を確認できます。
アップロード確認
キャッシュを任意でアップロードする方法はありません。そう……。
動きを見ている限りSwitch Platformするとアップロードが行われるようなので、まずはそこで動作確認してみます。
ブラウザでダッシュボードを開いてInternet Data Downloaded into cacheの値を見てから、Switch Platformを行って、数字が増えていたら成功です。
うまくアップロードできていれば、Switch Platformが早くなっているはずです。たぶん。Assets > ReImportでもアップロードしていそうな気配があるので、お好みでどうぞ。
追記として、まだPreviewですが、6だと右下のアイコンから任意でアップロードできるようになっていました。6000.0.22で確認しています。
ダウンロード確認
UnityEditorをいったん終了してLibraryフォルダを消して再起動します。
Editor.log上に以下のようにキャッシュサーバからDLしていそうなログが出ていれば成功です。キャッシュサーバへのアップロードが間に合っていないことがあるので、だめそうなら2~3回繰り返してください。
// こういうログとか
Querying for cacheable assets in Cache Server:
// こういうログが出ていればOKっぽい
ShaderCacheRemote downloaded 1932 bytes for key 'a9263d83c8358e7cab4a0ef7b4743c95'
Accelerator側のログをVerboseにして見ていると、以下がアップロード/ダウンロードのログっぽいです。
// アップロード
"msg":"handled putRequest"
// ダウンロード
"msg":"handled getRequest"
CLI
-adb2 -EnableCacheServer -cacheServerEndpoint localhost:5678
CLIから使う際は上記のような指定を追加します。必要な場合は-cacheServerEndpointの引数である接続先、localhost:5678の部分を変更してください。エディタでCheck ConnectionでSuccessするアドレスをそのまま入れれば大丈夫です。
-adb2はマニュアルに記載がないですが、ないと動かないという声がフォーラムに蔓延していたのでつけておけばいいと思います。
更に深く動かす人向け
https://docs.unity3d.com/2023.2/Documentation/Manual/UnityAccelerator.html#command
You can also run unity-accelerator --all-help to output all the help text for all commands in one go. It’s probably best to pipe the output through less or redirect output to a text file to read.
unity-accelerator内部のCLIでもっといろいろできるらしいです。
興味がある人はコンテナの中に以下のようなコマンドで入って触ってみるのもいいと思います。
// docker
docker-compose exec accelerator bash
// nerdctl
nerdctl exec accelerator bash
/usr/local/bin/unity-acceleratorにバイナリがありますが、パスは通っているのでどこでもコマンドを実行できます。
とはいえ見に行くのもたいへんだと思うので、unity-accelerator --all-helpの結果を以下に貼っておきます。
unity-accelerator --all-help
root@accelerator:/# unity-accelerator --all-help
Unity Accelerator v1.0.946+g7cc8240
Run on a local network to speed up transactions with Unity Services.
Most users will use "unity-accelerator register <token>" for the first time and then "unity-accelerator run" from then on.
The other tools are for advanced debugging.
Options:
-? -h --help Output this help text.
--all-help Output this help text and the help text for all subcommands.
--color Controls color output; use --no-color to disable. Default: if terminal
--version Output version information.
Subcommands:
cache Tools for working with actual cache contents.
config Tools for working with the unity-accelerator.cfg file.
dashboard Tools for working with the built-in dashboard website.
install Install the accelerator using built-in routines.
register Register the agent for the first time.
run Run the agent.
support-report Gather information for a report to Unity Support.
tool Debugging tools.
uninstall Uninstall the accelerator using built-in routines.
unregister Unregister an agent.
---[ unity-accelerator cache ]-------------------------------------------------------------------------------------------------------------
unity-accelerator cache <subcommand> [args]...
The cache subcommand contains tools for working with the actual cache contents.
Options:
-? -h --help Output this help text.
--all-help Output this help text and the help text for all subcommands.
Subcommands:
delete Delete an item from the cache.
delete-namespace Delete all items under a namespace from the cache.
get Get an item from the cache.
head Check the cache for an item's existence.
info Send an info request to the cache.
keys-since Lists keys in use in the cache, within a namespace, created after a timestamp.
list-namespaces Lists namespaces in use in the cache.
perf Tools for quick cache performance tests.
pull Retrieve cached data from another agent or agents.
push Send all cached data to another agent.
put Put an item into the cache.
---[ unity-accelerator cache delete ]------------------------------------------------------------------------------------------------------
unity-accelerator cache delete ip:port <namespace> <key>
Delete an item from the cache.
Options:
-? -h --help Output this help text.
---[ unity-accelerator cache delete-namespace ]--------------------------------------------------------------------------------------------
unity-accelerator cache delete-namespace ip:port <namespace>
Delete all items under a namespace from the cache.
Options:
-? -h --help Output this help text.
---[ unity-accelerator cache get ]---------------------------------------------------------------------------------------------------------
unity-accelerator cache get ip:port <namespace> <key>
Get an item from the cache.
Options:
-? -h --help Output this help text.
---[ unity-accelerator cache head ]--------------------------------------------------------------------------------------------------------
unity-accelerator cache head ip:port <namespace> <key>
Checks the cache for the existence of a value for namespace+key.
Output will be the size of the stored value or an error.
Options:
-? -h --help Output this help text.
--full-info Show full information returned by the Head request, in one line per 'key: value' format.
---[ unity-accelerator cache info ]--------------------------------------------------------------------------------------------------------
unity-accelerator cache info ip:port
Sends an info request to the cache server and outputs the response.
Options:
-? -h --help Output this help text.
---[ unity-accelerator cache keys-since ]--------------------------------------------------------------------------------------------------
unity-accelerator cache keys-since ip:port <namespace> <marker> <timestamp>
Lists keys in use in the cache, within a namespace, created after a marker+timestamp. Because keys could have special characters within
their names, the output will be in JSON format, one JSON object per key+timestamp.
The marker is the last key known with the timestamp given. For example, if there was run done yesterday and the last key+timestamp
returned was keyA+12345 then the next keys-since command would use keyA as the marker and 12345 as the timestamp. This is because there is
the possibility that a keyB was created (for example) with the exact same timestamp.
Options:
-? -h --help Output this help text.
---[ unity-accelerator cache list-namespaces ]---------------------------------------------------------------------------------------------
unity-accelerator cache list-namespaces ip:port
Lists namespaces in use in the cache. Because namespaces could have special characters within their names, the output will be in JSON
format, one JSON object per namespace name.
Options:
-? -h --help Output this help text.
---[ unity-accelerator cache perf ]--------------------------------------------------------------------------------------------------------
unity-accelerator cache perf <subcommand> [args]...
The perf subcommand contains tools for quick cache performance tests. For more intricate and higher load performance testing, see "tool
load-test" and "tool load-test-client".
To test basic network connectivity without actually altering the cache items stored:
unity-accelerator cache perf put <ip:port>
unity-accelerator cache perf get <ip:port>
These will each send 10,000 requests using the memory-discard namespace, which throws away put data and generates random get data. If they
run too quickly, you can add an --at-least 15s option or increase the count with the --count n option.
For a true cached-to-disk test, first put the data:
unity-accelerator cache perf put <ip:port> my-test-namespace
Then get the data:
unity-accelerator cache perf get <ip:port> my-test-namespace
The gets might go too fast as compared to the puts, so you might want to use the --at-least 15s option. Or you can increase the count on
the gets; however, you will have to use the --max-key n option to cause the gets to rotate the keys within those created with the earlier
puts.
unity-accelerator cache perf get <ip:port> my-test-namespace --count 50000 --max-key 10000
There we asked for 50,000 requests but kept the keys capped at 10,000 since the default puts we'd performed earlier only put 10,000 keys.
Options:
-? -h --help Output this help text.
--all-help Output this help text and the help text for all subcommands.
Subcommands:
get Performance test get requests.
head Performance test head requests.
put Performance test put requests.
---[ unity-accelerator cache perf get ]----------------------------------------------------------------------------------------------------
unity-accelerator cache perf get ip:port [namespace]
Performance test get requests. If [namespace] is omitted, the "memory-discard" namespace will be used.
The keys to get within the namespace will be the sequence number of each request. In other words, if you ran with --count 3 the keys would
be "1", "2", and "3". Use --max-key n to set a maximum key value -- key counts above this value will rotate back to "1".
The "memory-discard" namespace on the service-side does not actually use disk i/o to read the values; instead the service just generates
random content and is really just for testing network connectivity. Each memory-discard get request will generate the number of bytes
indicated by its key. For example, 1 byte will be served when retrieving the "1" key, 2 bytes for the "2" key, etc.
Options:
-? -h --help Output this help text.
--at-least d Runs for at least the duration given. Use durations like "10s" or "5m". If the test completes the --count
given before the --at-least time limit, it will just rerun through the same count again.
--concurrency n Maximum number of concurrent requests. Default: 100
--connections n Maximum number of concurrent connections. Default: 10
--count n Number of requests to perform. Default: 10000
--errors Output any error messages.
--json Output results encoded with JSON.
--max-key n The keys retrieved are the sequence number of the requests made, e.g. "1", "2", etc. This max-key option will
reset the sequence number at that limit, e.g. "--max-key 2" would cause "1", "2", "1", "2", etc.
--timeout d Time to wait for responses before returning errors. Use durations like "10s" or "5m". Default: 10s
---[ unity-accelerator cache perf head ]---------------------------------------------------------------------------------------------------
unity-accelerator cache perf head ip:port [namespace]
Performance test head requests. If [namespace] is omitted, the "memory-discard" namespace will be used.
The keys to head within the namespace will be the sequence number of each request. In other words, if you ran with --count 3 the keys
would be "1", "2", and "3". Use --max-key n to set a maximum key value -- key counts above this value will rotate back to "1".
The "memory-discard" namespace on the service-side does not actually use disk i/o to head the values; instead the service just pretends
all integer keys exist.
Options:
-? -h --help Output this help text.
--at-least d Runs for at least the duration given. Use durations like "10s" or "5m". If the test completes the --count
given before the --at-least time limit, it will just rerun through the same count again.
--concurrency n Maximum number of concurrent requests. Default: 100
--connections n Maximum number of concurrent connections. Default: 10
--count n Number of requests to perform. Default: 10000
--errors Output any error messages.
--json Output results encoded with JSON.
--max-key n The keys retrieved are the sequence number of the requests made, e.g. "1", "2", etc. This max-key option will
reset the sequence number at that limit, e.g. "--max-key 2" would cause "1", "2", "1", "2", etc.
--timeout d Time to wait for responses before returning errors. Use durations like "10s" or "5m". Default: 10s
---[ unity-accelerator cache perf put ]----------------------------------------------------------------------------------------------------
unity-accelerator cache perf put ip:port [namespace]
Performance test put requests. If [namespace] is omitted, the "memory-discard" namespace will be used. The "memory-discard" namespace will
not actually save any data to disk, so you won't have to worry about filling up disk space or evicting good cached data, but it's really
just testing network connectivity.
The keys put within the namespace will be the sequence number of each request. In other words, if you ran with --count 3 the keys would be
"1", "2", and "3".
Options:
-? -h --help Output this help text.
--at-least d Runs for at least the duration given. Use durations like "10s" or "5m". If the test completes the --count
given before the --at-least time limit, it will just rerun through the same count again.
--concurrency n Maximum number of concurrent requests. Default: 100
--connections n Maximum number of concurrent connections. Default: 10
--count n Number of requests to perform. Default: 10000
--errors Output any error messages.
--json Output results encoded with JSON.
--size n Size of each value. Default: 131072
--timeout d Time to wait for responses before returning errors. Use durations like "10s" or "5m". Default: 10s
---[ unity-accelerator cache pull ]--------------------------------------------------------------------------------------------------------
unity-accelerator cache pull ip:port <remote-endpoint> ...
Retrieve cached data from another agent or agents. The ip:port is the agent that will receive the data, the <remote-endpoint> is the agent
the data will be retrieved from -- you may include more than one <remote-endpoint>.
The <remote-endpoint> is in the format protobuftls://host:port/namespace where the tls and /namespace parts are optional. For example,
protobuf://1.2.3.4:10080 will pull all keys from all namespaces on the agent listening at ip 1.2.3.4, port 10080; whereas
protobuftls://5.6.7.8:10443/amazingproject will pull all keys from the amazingproject namespace on the agent listening at ip 5.6.7.8, port
10443.
The data retrieved will be the data newer than was retrieved previously. Once the pull has completed, the newest item pulled will be
recorded in the recipient's database, to track where to start for a possible future pull. You can either configure a scheduled job to call
this pull command to keep an agent up to date, or you can configure the agent to pull data for itself (see
unity-accelerator config help ProtobufRetrieveItems for more information).
Options:
-? -h --help Output this help text.
--agent-jwt s Use the agent-jwt auth procedures with the endpoint given; usually something like
https://host:port/api/agent-jwt Default: $UNITY_ACCELERATOR_CLIENT_AGENT_JWT
--debug Enable debug output. Default: $UNITY_ACCELERATOR_CLIENT_DEBUG
--insecure-skip-verify Ignore cert verification failures from TLS/SSL. Default: $UNITY_ACCELERATOR_CLIENT_INSECURE_SKIP_VERIFY
--tls --ssl Use TLS/SSL. Default: $UNITY_ACCELERATOR_CLIENT_TLS
--user s User to authenticate as, if needed. Format: env/login/project/url -- common example:
/user@host.com/64fc932d-7a88-4c00-acc2-4755a73fe0bf/ Default: $UNITY_ACCELERATOR_CLIENT_USER
---[ unity-accelerator cache push ]--------------------------------------------------------------------------------------------------------
unity-accelerator cache push ip:port ip:port
Send all cached data to another agent. The first ip:port is the source agent, the second is the destination.
All namespaces, keys, and values will be sent to the destination. The destination will accept any data that it is missing or that is newer
than what it already has.
The destination must be configured to accept Put requests from the source, and the source must be configured to allow sending Put requests
to the destination.
Options:
-? -h --help Output this help text.
--agent-jwt s Use the agent-jwt auth procedures with the endpoint given; usually something like
https://host:port/api/agent-jwt Default: $UNITY_ACCELERATOR_CLIENT_AGENT_JWT
--debug Enable debug output. Default: $UNITY_ACCELERATOR_CLIENT_DEBUG
--insecure-skip-verify Ignore cert verification failures from TLS/SSL. Default: $UNITY_ACCELERATOR_CLIENT_INSECURE_SKIP_VERIFY
--tls --ssl Use TLS/SSL. Default: $UNITY_ACCELERATOR_CLIENT_TLS
--user s User to authenticate as, if needed. Format: env/login/project/url -- common example:
/user@host.com/64fc932d-7a88-4c00-acc2-4755a73fe0bf/ Default: $UNITY_ACCELERATOR_CLIENT_USER
---[ unity-accelerator cache put ]---------------------------------------------------------------------------------------------------------
unity-accelerator cache put ip:port <namespace> <key> [file]
Put an item into the cache. The [file] is optional; if omitted stdin will be used.
Options:
-? -h --help Output this help text.
--blob-hash s Will send a specific value as the blob hash, rather than calculating it or just sending 0.
--skip-blob-hash Will skip calculating and sending the blob hash of the content; always true when receiving content from stdin.
---[ unity-accelerator config ]------------------------------------------------------------------------------------------------------------
unity-accelerator config <subcommand> [args]...
The config subcommand contains tools for working with the unity-accelerator.cfg file.
Options:
-? -h --help Output this help text.
--all-help Output this help text and the help text for all subcommands.
Subcommands:
get Get configuration values.
help Output detailed information about config fields.
set Set configuration values.
---[ unity-accelerator config get ]--------------------------------------------------------------------------------------------------------
unity-accelerator config get <config_value_name> [config_value_name] ...
Outputs the value for each config_value_name, retrieved from the unity-accelerator.cfg file stored in the --persist directory, or the
current working directory by default.
You may also use the --default option to output the default value for a configuration item, ignoring any existing config file.
There is one special case name, "persist", that is not stored on the unity-accelerator.cfg file but is available to output the directory
where the unity-accelerator.cfg is (or would be) located.
Options:
-? -h --help Output this help text.
--base64 Base64 encode each value.
--default Output default values instead of what is stored in the configuration file.
--ignore-missing Ignore a missing unity-accelerator.cfg file or persist directory.
--names Output each field name as well as its value.
--new-lines Output a new line after each value; use --no-new-lines to suppress. Default: true
--persist s Path where persisted data is stored; i.e. where the unity-accelerator.cfg file is. Default:
$UNITY_ACCELERATOR_PERSIST, .
---[ unity-accelerator config help ]-------------------------------------------------------------------------------------------------------
unity-accelerator config help [field-name] ...
Returns detailed information about the configuration fields named, or all fields if none were named.
Options:
-? -h --help Output this help text.
--persist s Path where persisted data is stored; i.e. where the unity-accelerator.cfg file is. Default: $UNITY_ACCELERATOR_PERSIST, .
---[ unity-accelerator config set ]--------------------------------------------------------------------------------------------------------
unity-accelerator config set <name> <value>
Updates the value of a configuration item to the value given.
For example: unity-accelerator config set CacheNoAccessTimes true
If you need to specify a value that starts with a minus sign, you'll need to use the Unix standard double-dash syntax to tell the command
line parser you will have no further options. This is probably clearer with examples:
unity-accelerator config set CacheMaxPending -1
That fails with: unknown option "--1"
So, to work around that you'd use the double-dash syntax:
unity-accelerator config set CacheMaxPending -- -1
Options:
-? -h --help Output this help text.
--persist s Path where persisted data is stored; i.e. where the unity-accelerator.cfg file is. Default: $UNITY_ACCELERATOR_PERSIST, .
---[ unity-accelerator dashboard ]---------------------------------------------------------------------------------------------------------
unity-accelerator dashboard <subcommand> [args]...
The dashboard subcommand contains tools related to the built-in dashboard website for the accelerator.
Options:
-? -h --help Output this help text.
--all-help Output this help text and the help text for all subcommands.
Subcommands:
list List dashboard accounts.
password Set the password for a dashboard account.
remove Remove a dashboard account.
url Get the url for the built-in dashboard.
---[ unity-accelerator dashboard list ]----------------------------------------------------------------------------------------------------
unity-accelerator dashboard list
Lists stored dashboard accounts.
Options:
-? -h --help Output this help text.
--persist s Path where persisted data is stored; i.e. where the unity-accelerator.cfg file is. Default: $UNITY_ACCELERATOR_PERSIST, .
---[ unity-accelerator dashboard password ]------------------------------------------------------------------------------------------------
unity-accelerator dashboard password <account>
Sets the password for a dashboard account, creating the account if it does not already exist.
This will prompt for the password, hiding the input.
You may also use the --password option to include the password on the command line, though this is less secure (shows in process list,
command line history, etc.)
Options:
-? -h --help Output this help text.
--password s Uses the string as the password rather than prompting for it.
--persist s Path where persisted data is stored; i.e. where the unity-accelerator.cfg file is. Default: $UNITY_ACCELERATOR_PERSIST,
.
---[ unity-accelerator dashboard remove ]--------------------------------------------------------------------------------------------------
unity-accelerator dashboard remove <account>
Removes the dashboard account, if it exists.
Options:
-? -h --help Output this help text.
--persist s Path where persisted data is stored; i.e. where the unity-accelerator.cfg file is. Default: $UNITY_ACCELERATOR_PERSIST, .
---[ unity-accelerator dashboard url ]-----------------------------------------------------------------------------------------------------
unity-accelerator dashboard url
Returns the URL for the built-in dashboard.
Options:
-? -h --help Output this help text.
--json Output details as JSON objects rather than plain text.
--persist s Path where persisted data is stored; i.e. where the unity-accelerator.cfg file is. Default: $UNITY_ACCELERATOR_PERSIST, .
---[ unity-accelerator install ]-----------------------------------------------------------------------------------------------------------
unity-accelerator install <install.json>
Install the accelerator using built-in routines. Recommended steps:
* Run unity-accelerator install --sample install.json to generate a sample install.json file with all the defaults.
* Edit the install.json to reflect your preferences.
* Run unity-accelerator install --dry-run install.json and evaluate the actions the installer would take.
* When satisfied, run unity-accelerator install install.json to finalize the installation.
Note that if the <install.json> filename is - then the install data will be read from stdin as a single JSON object.
Options:
-? -h --help Output this help text.
--dry-run Pretend to run the installation, outputting detail on the actions that would be taken.
--json Output progress details as JSON objects rather than plain text.
--sample Generate a sample install.json file.
--skip-admin-check Will skip the admin/root check and continue running as if it had such privileges.
---[ unity-accelerator register ]----------------------------------------------------------------------------------------------------------
unity-accelerator register <type> [token]
Register the agent for the first time.
<type> can be "collab", "adbv2", or "legacy" whichever you'd like to register and enable. If you'd like multiple, run register once with
each <type>.
[token] is the registration token when registering with "collab"; the token can be found on the developer dashboard accelerator page.
Examples:
unity-accelerator register collab Jx39n7
unity-accelerator register adbv2
unity-accelerator register legacy
Options:
-? -h --help Output this help text.
--debug Enable debug output. Use --no-debug to disable. Default: $UNITY_ACCELERATOR_DEBUG
--log-stdout Logs to stdout rather than handling log files and rotations itself. Default: $UNITY_ACCELERATOR_LOG_STDOUT
--log-stdout-also Logs to stdout as well as handling log files and rotations itself. Default: $UNITY_ACCELERATOR_LOG_STDOUT_ALSO
--persist s Path where persisted data is stored; i.e. where the unity-accelerator.cfg file is. Default:
$UNITY_ACCELERATOR_PERSIST, .
---[ unity-accelerator run ]---------------------------------------------------------------------------------------------------------------
unity-accelerator run
Runs the agent, starting up any configured services for use.
--[ Process Management Details ]--
Once running, you may notice two processes: the run process initially launched and a tool wrun child process also launched. run is
just a simple managing process that launches tool wrun with the exact same command line options. This two-process configuration is
done to facilitate automatic upgrades, dashboard restarts, etc.
While running, the service may download a new version of itself, which it will store as unity-accelerator-next (with .exe extension on
Windows) in the persist directory (alongside the unity-accelerator.cfg file, cachedb dir, etc.).
During the configured maintenance window (midnight by default), the child process tool wrun will exit and the parent process run will
rename unity-accelerator-next to unity-accelerator and launch that with tool wrun once again. It will also keep a
unity-accelerator-previous around if there already was a unity-accelerator in place.
In this way, the parent process run never exits (unless deliberately asked to shutdown) and therefore also runs the probably older
installed version of unity-accelerator. However, the part of the code that run uses is quite small, mostly limited to checking for new
versions and just launching tool wrun.
If you do not want automatic upgrades you can turn them off with config set NoAutoUpdates true. Be sure to remove any
unity-accelerator or unity-accelerator-next files from your persist directory as well, or they will be used by run to launch
tool wrun.
Options:
-? -h --help Output this help text.
--auto-updates Indicates whether automatic updates should be enabled. Use --no-auto-updates to disable. Default:
$UNITY_ACCELERATOR_AUTO_UPDATES
--debug Enable debug output. Use --no-debug to disable. Default: $UNITY_ACCELERATOR_DEBUG
--http-port n Port to listen on for HTTP requests; this will override all other methods to set the port. Default:
$UNITY_ACCELERATOR_HTTP_PORT
--log-stdout Logs to stdout rather than handling log files and rotations itself. Default: $UNITY_ACCELERATOR_LOG_STDOUT
--log-stdout-also Logs to stdout as well as handling log files and rotations itself. Default: $UNITY_ACCELERATOR_LOG_STDOUT_ALSO
--persist s Path where persisted data is stored; i.e. where the unity-accelerator.cfg file is. Default:
$UNITY_ACCELERATOR_PERSIST, .
--protobuf-port n Port to listen on for protobuf requests; this will override all other methods to set the port. Default:
$UNITY_ACCELERATOR_PROTOBUF_PORT
---[ unity-accelerator support-report ]----------------------------------------------------------------------------------------------------
unity-accelerator support-report <filename.zip>
Gather information for a report to Unity Support.
This will create the <filename.zip> as a zip archive file to send to Unity Support. This file will contain:
* The unity-accelerator.cfg with sensitive fields redacted.
* The last ten megabytes of the unity-accelerator.log and unity-accelerator.run.log files.
* The unity-accelerator.metrics file (contains a snapshot of the http /metrics endpoint).
Options:
-? -h --help Output this help text.
--persist s Path where persisted data is stored; i.e. where the unity-accelerator.cfg file is. Default: $UNITY_ACCELERATOR_PERSIST, .
---[ unity-accelerator tool ]--------------------------------------------------------------------------------------------------------------
unity-accelerator tool <subcommand> [args]...
Advanced debugging tools.
WARNING: These commands and their arguments may change or be removed in future releases.
Options:
-? -h --help Output this help text.
--all-help Output this help text and the help text for all subcommands.
Subcommands:
bbolt Debug access to the bbolt.db (cache tracking).
blob-hash Calculate the blob hash for content.
cdp Post JSON data to CDP endpoint.
cert TLS certificate commands.
delete-multi Delete multiple items from cache.
get-multi Get multiple items from cache.
head-multi Check for value existence of multiple items.
load-test Load testing coordinator.
load-test-client Load testing client.
put-multi Put multiple items into cache.
wrun Run the agent (called by "unity-accelerator run" usually).
---[ unity-accelerator tool bbolt ]--------------------------------------------------------------------------------------------------------
unity-accelerator tool bbolt <path-to-bbolt.db>
Dumps information from <path-to-bbolt.db>.
Options:
-? -h --help Output this help text.
---[ unity-accelerator tool blob-hash ]----------------------------------------------------------------------------------------------------
unity-accelerator tool blob-hash <file>
Calculates the blob hash (XXH64) for the content of <file>, you may use a single dash - for <file> to represent stdin.
Options:
-? -h --help Output this help text.
---[ unity-accelerator tool cdp ]----------------------------------------------------------------------------------------------------------
unity-accelerator tool cdp <custom_endpoint> <cmd> <args> (send to custom endpoint)
unity-accelerator tool cdp default <cmd> <args> (use the default defined endpoint: https://prd-lender.cdp.internal.unity3d.com)
unity-accelerator tool cdp test <cmd> <args> (to test without sending to an endpoint)
Subcommands:
InstallerDownloaded <os> <download_location> <org_id> <user_id> <env>
InstallerStatus <os> <launched> <uninstalled> <env>
InstallerCompletion <os> <completed> <error_type> <env> <collab_enabled> <adbv2_enabled> <legacy_enabled>
Options:
-? -h --help Output this help text.
--persist s Path where persisted data is stored; i.e. where the unity-accelerator.cfg file is. Default: $UNITY_ACCELERATOR_PERSIST, .
---[ unity-accelerator tool cert ]---------------------------------------------------------------------------------------------------------
unity-accelerator tool cert { gen | use } <hostname> <cert.pem> <key.pem>
Generate or use a TLS certificate.
Options:
-? -h --help Output this help text.
--persist s Path where persisted data is stored; i.e. where the unity-accelerator.cfg file is. Default: $UNITY_ACCELERATOR_PERSIST, .
---[ unity-accelerator tool delete-multi ]-------------------------------------------------------------------------------------------------
unity-accelerator tool delete-multi ip:port <namespace> <key1> [key2]...
delete-multi will delete each key's value from the cache.
Options:
-? -h --help Output this help text.
--agent-jwt s Use the agent-jwt auth procedures with the endpoint given; usually something like
https://host:port/api/agent-jwt Default: $UNITY_ACCELERATOR_CLIENT_AGENT_JWT
--debug Enable debug output. Default: $UNITY_ACCELERATOR_CLIENT_DEBUG
--insecure-skip-verify Ignore cert verification failures from TLS/SSL. Default: $UNITY_ACCELERATOR_CLIENT_INSECURE_SKIP_VERIFY
--tls --ssl Use TLS/SSL. Default: $UNITY_ACCELERATOR_CLIENT_TLS
--user s User to authenticate as, if needed. Format: env/login/project/url -- common example:
/user@host.com/64fc932d-7a88-4c00-acc2-4755a73fe0bf/ Default: $UNITY_ACCELERATOR_CLIENT_USER
---[ unity-accelerator tool get-multi ]----------------------------------------------------------------------------------------------------
unity-accelerator tool get-multi ip:port <namespace> <key1> [key2]...
get-multi will download each key's value to a file whose name is the key; use --dst <path> to indicate where the files will be created.
Options:
-? -h --help Output this help text.
--agent-jwt s Use the agent-jwt auth procedures with the endpoint given; usually something like
https://host:port/api/agent-jwt Default: $UNITY_ACCELERATOR_CLIENT_AGENT_JWT
--debug Enable debug output. Default: $UNITY_ACCELERATOR_CLIENT_DEBUG
--discard Discards all downloaded data; for testing.
--dst s Path where downloaded data will be stored. Default: .
--insecure-skip-verify Ignore cert verification failures from TLS/SSL. Default: $UNITY_ACCELERATOR_CLIENT_INSECURE_SKIP_VERIFY
--tls --ssl Use TLS/SSL. Default: $UNITY_ACCELERATOR_CLIENT_TLS
--user s User to authenticate as, if needed. Format: env/login/project/url -- common example:
/user@host.com/64fc932d-7a88-4c00-acc2-4755a73fe0bf/ Default: $UNITY_ACCELERATOR_CLIENT_USER
---[ unity-accelerator tool head-multi ]---------------------------------------------------------------------------------------------------
unity-accelerator tool head-multi ip:port <namespace> <key1> [key2]...
head-multi will check for the existence of each key.
Each line output will be "namespace key size" or an error for each key.
Options:
-? -h --help Output this help text.
--agent-jwt s Use the agent-jwt auth procedures with the endpoint given; usually something like
https://host:port/api/agent-jwt Default: $UNITY_ACCELERATOR_CLIENT_AGENT_JWT
--debug Enable debug output. Default: $UNITY_ACCELERATOR_CLIENT_DEBUG
--insecure-skip-verify Ignore cert verification failures from TLS/SSL. Default: $UNITY_ACCELERATOR_CLIENT_INSECURE_SKIP_VERIFY
--tls --ssl Use TLS/SSL. Default: $UNITY_ACCELERATOR_CLIENT_TLS
--user s User to authenticate as, if needed. Format: env/login/project/url -- common example:
/user@host.com/64fc932d-7a88-4c00-acc2-4755a73fe0bf/ Default: $UNITY_ACCELERATOR_CLIENT_USER
---[ unity-accelerator tool load-test ]----------------------------------------------------------------------------------------------------
unity-accelerator tool load-test <loadtest.cfg>
NOTE: This load test infrastructure does not [yet] support calculating, sending, or validating blob hashes.
The load-test command will manage load testing jobs, which are run on load-test-client instances, gather and report the results.
Use the --create option to create a new <loadtest.cfg> file with default values as a starting point for your own load testing.
Let's say you edit this fresh cfg file and make sure the list of clients, the target, etc. are accurate. You would then just run:
unity-accelerator tool load-test loadtest.cfg
This would launch the load test controller for that configuration, which would launch the needed jobs on the load testing clients, which
would start making requests against the target. The output might look something like this:
{"level":"info","ts":"2020-04-30T12:58:29.188-0700","msg":"put jobs started","test_id":"loadtest_171ccab4743"}
{"level":"info","ts":"2020-04-30T12:59:29.188-0700","msg":"put jobs progress","test_id":"loadtest_171ccab4743","elapsed":60.00028329,"count":30570,"success_count":30569,"success_count_persec":509.48092781913266,"byte_count":5802937633,"byte_count_persec":96715170.5759888,"error_count":0,"error_count_persec":0}
...
{"level":"info","ts":"2020-04-30T13:00:44.193-0700","msg":"put jobs complete","test_id":"loadtest_171ccab4743","elapsed":135.005700836,"count":100000,"success_count":100000,"success_count_persec":740.7094617543326,"byte_count":19006045440,"byte_count_persec":140779576.87940788,"error_count":0,"error_count_persec":0}
{"level":"info","ts":"2020-04-30T13:00:44.193-0700","msg":"get jobs started","test_id":"loadtest_171ccab4743"}
{"level":"info","ts":"2020-04-30T13:01:44.194-0700","msg":"get jobs progress","test_id":"loadtest_171ccab4743","elapsed":60.000397958,"get_job_count":10,"get_jobs_left":10,"count":154914,"success_count":154274,"success_count_persec":2571.2162793985312,"byte_count":29451633139,"byte_count_persec":490857296.64019907,"missing_count":0,"missing_count_persec":0,"error_count":0,"error_count_persec":0}
...
{"level":"info","ts":"2020-04-30T13:05:44.256-0700","msg":"get jobs complete","test_id":"loadtest_171ccab4743","elapsed":300.062886883,"get_job_count":10,"count":1000000,"success_count":1000000,"success_count_persec":3332.634736630786,"byte_count":190060454400,"byte_count_persec":633402072.3932716,"missing_count":0,"missing_count_persec":0,"error_count":0,"error_count_persec":0}
Options:
-? -h --help Output this help text.
--create Create the config file with default values, if the file does not already exist.
--debug Output debug information.
---[ unity-accelerator tool load-test-client ]---------------------------------------------------------------------------------------------
unity-accelerator tool load-test-client <addr>
Start a load testing client listening on the addr given; addr is "[ip]:port". For example, ":0" means "all ips, pick a random port".
":12345" means "all ips, port 12345. And "127.0.0.1:12345" means "only listen on 127.0.0.1 on port 12345".
A load testing client is a simple HTTP server that accepts requests to launch load testing jobs. A load testing controller would normally
be used to contact one or more clients to launch one or more load testing jobs against a single running cache server.
For example, you could start a client with:
unity-accelerator tool load-test-client :12345
And then issue an adhoc load testing job request with:
curl -NX POST http://127.0.0.1:12345/job -d '{"job":"put","target":"127.0.0.1:10080","span":"2m"}'
Which will then launch a load testing "put job" that sends data to the "target" as fast as it can for "2m" (two minutes) and uses whatever
the defaults are otherwise (e.g. each put is 4096 bytes, single concurrency, etc.)
While the job is running, curl will receive json encoded progress updates, such as:
{"job_id":1,"msg":"launched put job","ts":"2020-04-30T11:13:39-07:00"}
{"byte_count":91619328,"error_count":0,"job_id":1,"msg":"put job progress","put_count":22368,"success_count":22368,"ts":"2020-04-30T11:13:54-07:00"}
{"byte_count":222015488,"error_count":0,"job_id":1,"msg":"put job progress","put_count":54203,"success_count":54202,"ts":"2020-04-30T11:14:09-07:00"}
...
{"byte_count":1038864384,"error_count":0,"job_id":1,"msg":"put job progress","put_count":253629,"success_count":253629,"ts":"2020-04-30T11:15:39-07:00"}
{"job_id":1,"msg":"completed put job","ts":"2020-04-30T11:15:39-07:00"}
Usually the load testing clients are used by a load testing controller, rather than adhoc curl requests. For example:
unity-accelerator tool load-test --create loadtest.cfg
This creates a fresh loadtest.cfg with all the defaults and documentation on what changes what. Let's say you edit the cfg file and make
sure the list of clients, the target, etc. are accurate. You would then just run:
unity-accelerator tool load-test loadtest.cfg
This would launch the load test controller for that configuration, which would launch the needed jobs on the load testing clients, which
would start making requests against the target. The output might look something like this:
{"level":"info","ts":"2020-04-30T12:58:29.188-0700","msg":"put jobs started","test_id":"loadtest_171ccab4743"}
{"level":"info","ts":"2020-04-30T12:59:29.188-0700","msg":"put jobs progress","test_id":"loadtest_171ccab4743","elapsed":60.00028329,"count":30570,"success_count":30569,"success_count_persec":509.48092781913266,"byte_count":5802937633,"byte_count_persec":96715170.5759888,"error_count":0,"error_count_persec":0}
...
{"level":"info","ts":"2020-04-30T13:00:44.193-0700","msg":"put jobs complete","test_id":"loadtest_171ccab4743","elapsed":135.005700836,"count":100000,"success_count":100000,"success_count_persec":740.7094617543326,"byte_count":19006045440,"byte_count_persec":140779576.87940788,"error_count":0,"error_count_persec":0}
{"level":"info","ts":"2020-04-30T13:00:44.193-0700","msg":"get jobs started","test_id":"loadtest_171ccab4743"}
{"level":"info","ts":"2020-04-30T13:01:44.194-0700","msg":"get jobs progress","test_id":"loadtest_171ccab4743","elapsed":60.000397958,"get_job_count":10,"get_jobs_left":10,"count":154914,"success_count":154274,"success_count_persec":2571.2162793985312,"byte_count":29451633139,"byte_count_persec":490857296.64019907,"missing_count":0,"missing_count_persec":0,"error_count":0,"error_count_persec":0}
...
{"level":"info","ts":"2020-04-30T13:05:44.256-0700","msg":"get jobs complete","test_id":"loadtest_171ccab4743","elapsed":300.062886883,"get_job_count":10,"count":1000000,"success_count":1000000,"success_count_persec":3332.634736630786,"byte_count":190060454400,"byte_count_persec":633402072.3932716,"missing_count":0,"missing_count_persec":0,"error_count":0,"error_count_persec":0}
Options:
-? -h --help Output this help text.
--debug Enable debug output.
---[ unity-accelerator tool put-multi ]----------------------------------------------------------------------------------------------------
unity-accelerator tool put-multi ip:port <namespace> <file1> [file2]...
put-multi will upload each file under the given namespace and will use each file's name as its key.
Options:
-? -h --help Output this help text.
--agent-jwt s Use the agent-jwt auth procedures with the endpoint given; usually something like
https://host:port/api/agent-jwt Default: $UNITY_ACCELERATOR_CLIENT_AGENT_JWT
--debug Enable debug output. Default: $UNITY_ACCELERATOR_CLIENT_DEBUG
--insecure-skip-verify Ignore cert verification failures from TLS/SSL. Default: $UNITY_ACCELERATOR_CLIENT_INSECURE_SKIP_VERIFY
--skip-hash Will skip calculating and sending the hash of the content.
--tls --ssl Use TLS/SSL. Default: $UNITY_ACCELERATOR_CLIENT_TLS
--user s User to authenticate as, if needed. Format: env/login/project/url -- common example:
/user@host.com/64fc932d-7a88-4c00-acc2-4755a73fe0bf/ Default: $UNITY_ACCELERATOR_CLIENT_USER
---[ unity-accelerator tool wrun ]---------------------------------------------------------------------------------------------------------
unity-accelerator tool wrun
Runs the agent, starting up any configured services for use.
Note that unity-accelerator tool wrun is usually launched by run and not directly. See the help for the run command for more detail.
Options:
-? -h --help Output this help text.
--auto-updates Indicates whether automatic updates should be enabled. Use --no-auto-updates to disable. Default:
$UNITY_ACCELERATOR_AUTO_UPDATES
--debug Enable debug output. Use --no-debug to disable. Default: $UNITY_ACCELERATOR_DEBUG
--http-port n Port to listen on for HTTP requests; this will override all other methods to set the port. Default:
$UNITY_ACCELERATOR_HTTP_PORT
--log-stdout Logs to stdout rather than handling log files and rotations itself. Default: $UNITY_ACCELERATOR_LOG_STDOUT
--log-stdout-also Logs to stdout as well as handling log files and rotations itself. Default: $UNITY_ACCELERATOR_LOG_STDOUT_ALSO
--persist s Path where persisted data is stored; i.e. where the unity-accelerator.cfg file is. Default:
$UNITY_ACCELERATOR_PERSIST, .
--protobuf-port n Port to listen on for protobuf requests; this will override all other methods to set the port. Default:
$UNITY_ACCELERATOR_PROTOBUF_PORT
---[ unity-accelerator uninstall ]---------------------------------------------------------------------------------------------------------
unity-accelerator uninstall <uninstall.json>
NOTE: UNINSTALLING WILL REMOVE ALL CACHED DATA AS WELL AS CONFIGURATION!
Un-install the accelerator using built-in routines. After a previous install, there should be an uninstall.json file stored in your
<persist-dir>/runtime directory that contains information for what to remove during an un-install. It is recommended to run with --dry-run
first.
Note that if the <uninstall.json> filename is - then the uninstall data will be read from stdin as a single JSON object.
Options:
-? -h --help Output this help text.
--dry-run Pretend to run the un-installation, outputting detail on the actions that would be taken.
--json Output progress details as JSON objects rather than plain text.
--skip-admin-check Will skip the admin/root check and continue running as if it had such privileges.
---[ unity-accelerator unregister ]--------------------------------------------------------------------------------------------------------
unity-accelerator unregister <type>
Unregister an agent.
<type> One of "collab", "adbv2", or "legacy", whichever you'd like to disable.
Options:
-? -h --help Output this help text.
--debug Enable debug output. Use --no-debug to disable. Default: $UNITY_ACCELERATOR_DEBUG
--log-stdout Logs to stdout rather than handling log files and rotations itself. Default: $UNITY_ACCELERATOR_LOG_STDOUT
--log-stdout-also Logs to stdout as well as handling log files and rotations itself. Default: $UNITY_ACCELERATOR_LOG_STDOUT_ALSO
--persist s Path where persisted data is stored; i.e. where the unity-accelerator.cfg file is. Default:
$UNITY_ACCELERATOR_PERSIST, .
root@accelerator:/#
その他雑記
- なにがどうアップロードされるのかハンドリングできないという時点で速度計測をするのはやめました
- ダッシュボードのMetricsとかいろいろ動かしてても0のままの項目があって本当に動いてるかどうか不安
-
unity-accelerator.cfgとかいう設定ファイルがあるらしいが、コンテナでは生成されない？ - 改行コードを統一しておかないとなんか問題が起きる？ らしい？
- CLIのIFを見る限り、どのパスのどのファイルがキャッシュされているかの一覧をダッシュボードで表示するのはできなくはない気がするが、プロジェクトによっては膨大になりすぎるか。でも見れたらうれしい
- フォーラムを見る限りなんかたまに動かなくなるらしいので、volume込でコンテナをふっとばせるのは気持ちが楽
- Acceleratorが設定されているのに繋がらないとビルドがエラー落ちすることがあるらしい
- Library以下をキャッシュできる
GithubActionsなどを使っているとそこまで嬉しくなさそう。プラットフォーム毎に分けてキャッシュするコスト<<<<<Acceleratorを立てるコスト - 社内で使うなら使っていないマシンをIP固定にしてキャッシュサーバとして余生を過ごしてもらうといいか？ ストレージや回線など条件によるか。
まとめ
なんか……大変じゃない……？ というのが正直な感想です。
とはいえ立ち上げて使うだけならこの記事のdocker-composeを適当に立ち上げてUnityEditorにアドレスいれるだけなのでだいぶハードルは下げられたと思います。ひとまず動いていると信じて使い続けて、なにかあったらこの記事を更新しようかな、という気持ち。
おしまい。
参考
公式マニュアル。
わかるような……わからないような……一次情報なので読むけれど……。
Acceleratorの使い方自体はこのあたりを参考に。
なんかいろいろ面倒だからComposeにしたらいいじゃん！ という気付きをくれた。
なぜかProgress.Count()が0を返してきてうまくいかなかった。
Discussion