Compare commits

...

17 Commits

Author SHA1 Message Date
AK-Khan02
16633e3c65 Return 400 for missing /api/serialize IDs (#1272) 2026-06-10 18:48:33 -07:00
Floyd Horng
d12120bab6 Fix syncback not removing stale $properties at engine defaults (#1244) 2026-06-10 18:47:50 -07:00
boatbomber
ac6941f054 Add origin/host validation and warning for exposed serves (#1270) 2026-06-07 15:51:05 -07:00
Ken Loeffler
444dc11b26 Fix init.plugin.lua/init.plugin.luau not being supported (#1252) 2026-06-07 15:50:04 -07:00
EgoMoose
15939b1647 Add ignorable glob support (#1256) 2026-06-02 20:40:12 -07:00
AK-Khan02
0afb26e143 Update stale README and contributor documentation (#1269) 2026-06-02 09:11:41 -07:00
boatbomber
1abf675949 Graceful errors instead of crashing (#1267) 2026-06-01 20:02:39 -07:00
boatbomber
85655ca84f Heuristic hydration matching instead of hard dependency on GetChildren ordering (#1266)
### Summary

When two or more sibling instances share the same `Name` and
`ClassName`, Rojo's reconciler previously paired them with their
server-side counterparts purely by child order (first-unvisited match in
`GetChildren()` order). If the workspace child order ever diverged from
the server's, the wrong instance got paired so each duplicate could
inherit a sibling's properties.

This is the root of the #1257 bug: the welded parts would oscillate
between positions on each connect/disconnect because hydration kept
mis-pairing them. (#1265 stopped the sync fallback from scrambling child
order in the first place but this PR makes hydration robust even when
order *does* diverge.)

This PR makes `hydrate` break ties by comparing properties: when several
existing children match on `Name`+`ClassName`, it scores each candidate
by how many of the virtual instance's properties match the candidate's
live values, and picks the best. Order remains the tiebreak when scores
are equal, so behavior is unchanged for uniquely-named instances and for
indistinguishable siblings.

### Changes

- `trueEquals.lua`: extracted verbatim from `diff.lua` (the fuzzy
value-equality helper) so it can be shared. No behavior change;
`diff.lua` now requires it.
- `countMatchingProperties.lua`: added
`countMatchingProperties(instance, virtualInstance, instanceMap) ->
number`. Skips `Ref` properties (the instanceMap isn't fully built
mid-hydrate, so refs can't be decoded reliably, and they're a poor
disambiguator anyway) and any property that can't be read or decoded.
- `hydrate.lua`: See details below.

### Hydrate Changes

This touches `hydrate`, which runs over the whole tree on every
connect/resync, so I want state clearly that **the common path is faster
than before, not slower** even for parents with thousands of children!

The old algorithm was a nested scan: for each of `V` virtual children,
scan existing children until the first unvisited `Name`+`ClassName`
match. Two costs stand out:
- A `pcall` (to guard DataModel permission errors) ran on every
comparison (up to `V*E` `pcall`s per parent).
- Even for in-order trees the re-scanning of the visited prefix made it
`O(V^2)`.

The new algorithm does a single bucketing pass, then `O(1)` lookups:
1. One `O(E)` pass groups existing children into nested
`buckets[name][className]` tables. This runs exactly `E` `pcall`s total
(one per child), down from the `V*E` worst case.
2. Each virtual child does an `O(1)` bucket lookup to find its
candidates.
3. A per-bucket cursor skips already-paired children, so order-based
matching is amortized `O(1)` per child instead of rescanning.

| Scenario | Old | New |
| ------------------------------------------------ |
--------------------------------------- |
--------------------------------------------- |
| Unique-named children (typical, incl. thousands) | `O(V^2)`, plus up
to `V*E` pcalls | `O(V + E)`, plus exactly `E` pcalls |
| `C <= 32` candidates | `O(C^2)` | `O(P * C^2)` scoring |
| `C > 32` candidates | `O(C^2)` | `O(C)` |

Property scoring (`getProperty`/`decodeValue`/`trueEquals`) is the only
new expense, and it's gated two ways:
- It runs only when a `Name`+`ClassName` group has >=2 candidates (i.e.
never for uniquely-named instances).
- A cap, `MAX_CANDIDATES_TO_SCORE = 32`, means scoring only kicks in
once a group has <=32 unvisited candidates. A folder of thousands of
identically-named parts therefore does not trigger scoring; it falls
back to the original order-based pairing. The worst-case added scoring
work is bounded to roughly `32^2` property comparisons per group,
independent of group size.

So overall this is faster when you have unique names or many children.
It is slower but more robust when you have small groups of duplicate
names. Memory usage is increased as it creates the candidate buckets.
2026-06-01 17:29:22 -07:00
boatbomber
ae8735c80a Fix replaceInstances messing up GetChildren ordering (#1265)
Fixes #1257. replaceInstances now preserves GetChildren ordering.

I also had the unit tests unreliably say that Packages wasn't a member
of ReplicatedStorage so I added WaitForChild and tests run 100% for me
now.
2026-06-01 17:04:25 -07:00
Ken Loeffler
988efb45b1 Use lossy conversion for msgpack UInt64 decode in plugin (#1255) 2026-05-29 10:45:08 -07:00
Micah
9bbb1edd79 Upload plugin as part of release workflow (#1227) 2026-02-14 11:10:36 -08:00
Ivan Matthew
a2adf2b517 Improves sourcemap path handling with pathdiff (#1217) 2026-02-12 19:17:28 -08:00
Micah
4deda0e155 Use msgpack for API (#1176) 2026-02-12 18:37:24 -08:00
ari
4df2d3c5f8 Add actor, bindables and remotes to json_model_classes (#1199) 2026-02-12 17:34:32 -08:00
boatbomber
4965165ad5 Add option to forget prior info for place in reminder notif (#1215) 2026-01-23 21:15:34 +00:00
boatbomber
68eab3479a Fix notification unmount thread cancel bug (#1211) 2026-01-19 16:35:19 -08:00
Ivan Matthew
2a1102fc55 Implement VFS Path normalization for improved cross-platform tree synchronization (#1201) 2026-01-19 15:04:59 -08:00
98 changed files with 3005 additions and 456 deletions

View File

@@ -45,6 +45,13 @@ jobs:
name: Rojo.rbxm name: Rojo.rbxm
path: Rojo.rbxm path: Rojo.rbxm
- name: Upload Plugin to Roblox
env:
RBX_API_KEY: ${{ secrets.PLUGIN_UPLOAD_TOKEN }}
RBX_UNIVERSE_ID: ${{ vars.PLUGIN_CI_PLACE_ID }}
RBX_PLACE_ID: ${{ vars.PLUGIN_CI_UNIVERSE_ID }}
run: lune run upload-plugin Rojo.rbxm
build: build:
needs: ["create-release"] needs: ["create-release"]
strategy: strategy:

3
.gitignore vendored
View File

@@ -23,3 +23,6 @@
# Macos file system junk # Macos file system junk
._* ._*
.DS_STORE .DS_STORE
# JetBrains IDEs
/.idea/

6
.gitmodules vendored
View File

@@ -16,3 +16,9 @@
[submodule "plugin/Packages/Highlighter"] [submodule "plugin/Packages/Highlighter"]
path = plugin/Packages/Highlighter path = plugin/Packages/Highlighter
url = https://github.com/boatbomber/highlighter.git url = https://github.com/boatbomber/highlighter.git
[submodule "plugin/Packages/msgpack-luau"]
path = plugin/Packages/msgpack-luau
url = https://github.com/cipharius/msgpack-luau/
[submodule ".lune/opencloud-execute"]
path = .lune/opencloud-execute
url = https://github.com/Dekkonot/opencloud-luau-execute-lune.git

8
.lune/.config.luau Normal file
View File

@@ -0,0 +1,8 @@
return {
luau = {
languagemode = "strict",
aliases = {
lune = "~/.lune/.typedefs/0.10.4/",
},
},
}

View File

@@ -0,0 +1,51 @@
local args: any = ...
assert(args, "no arguments passed to script")
local input: buffer = args.BinaryInput
local AssetService = game:GetService("AssetService")
local SerializationService = game:GetService("SerializationService")
local EncodingService = game:GetService("EncodingService")
local input_hash: buffer = EncodingService:ComputeBufferHash(input, Enum.HashAlgorithm.Sha256)
local hex_hash: { string } = table.create(buffer.len(input_hash))
for i = 0, buffer.len(input_hash) - 1 do
table.insert(hex_hash, string.format("%02x", buffer.readu8(input_hash, i)))
end
print(`Deserializing plugin file (size: {buffer.len(input)} bytes, hash: {table.concat(hex_hash, "")})`)
local plugin = SerializationService:DeserializeInstancesAsync(input)[1]
local UploadDetails = require(plugin.UploadDetails) :: any
local PLUGIN_ID = UploadDetails.assetId
local PLUGIN_NAME = UploadDetails.name
local PLUGIN_DESCRIPTION = UploadDetails.description
local PLUGIN_CREATOR_ID = UploadDetails.creatorId
local PLUGIN_CREATOR_TYPE = UploadDetails.creatorType
assert(typeof(PLUGIN_ID) == "number", "UploadDetails did not contain a number field 'assetId'")
assert(typeof(PLUGIN_NAME) == "string", "UploadDetails did not contain a string field 'name'")
assert(typeof(PLUGIN_DESCRIPTION) == "string", "UploadDetails did not contain a string field 'description'")
assert(typeof(PLUGIN_CREATOR_ID) == "number", "UploadDetails did not contain a number field 'creatorId'")
assert(typeof(PLUGIN_CREATOR_TYPE) == "string", "UploadDetails did not contain a string field 'creatorType'")
assert(
Enum.AssetCreatorType:FromName(PLUGIN_CREATOR_TYPE) ~= nil,
"UploadDetails field 'creatorType' was not a valid member of Enum.AssetCreatorType"
)
print(`Uploading to {PLUGIN_ID}`)
print(`Plugin Name: {PLUGIN_NAME}`)
print(`Plugin Description: {PLUGIN_DESCRIPTION}`)
local result, version_or_err = AssetService:CreateAssetVersionAsync(plugin, Enum.AssetType.Plugin, PLUGIN_ID, {
["Name"] = PLUGIN_NAME,
["Description"] = PLUGIN_DESCRIPTION,
["CreatorId"] = PLUGIN_CREATOR_ID,
["CreatorType"] = Enum.AssetCreatorType:FromName(PLUGIN_CREATOR_TYPE),
})
if result ~= Enum.CreateAssetResult.Success then
error(`Plugin failed to upload because: {result.Name} - {version_or_err}`)
end
print(`Plugin uploaded successfully. New version is {version_or_err}.`)

78
.lune/upload-plugin.luau Normal file
View File

@@ -0,0 +1,78 @@
local fs = require("@lune/fs")
local process = require("@lune/process")
local stdio = require("@lune/stdio")
local luau_execute = require("./opencloud-execute")
local UNIVERSE_ID = process.env["RBX_UNIVERSE_ID"]
local PLACE_ID = process.env["RBX_PLACE_ID"]
local version_string = fs.readFile("plugin/Version.txt")
local versions = { string.match(version_string, "^v?(%d+)%.(%d+)%.(%d+)(.*)$") }
if versions[4] ~= "" then
print("This release is a pre-release. Skipping uploading plugin.")
process.exit(0)
end
local plugin_path = process.args[1]
assert(
typeof(plugin_path) == "string",
"no plugin path provided, expected usage is `lune run upload-plugin [PATH TO RBXM]`."
)
-- For local testing
if process.env["CI"] ~= "true" then
local rojo = process.exec("rojo", { "build", "plugin.project.json", "--output", plugin_path })
if not rojo.ok then
stdio.ewrite("plugin upload failed because: could not build plugin.rbxm\n\n")
stdio.ewrite(rojo.stderr)
stdio.ewrite("\n")
process.exit(1)
end
else
assert(fs.isFile(plugin_path), `Plugin file did not exist at {plugin_path}`)
end
local plugin_content = fs.readFile(plugin_path)
local engine_script = fs.readFile(".lune/scripts/plugin-upload.luau")
print("Creating task to upload plugin")
local task = luau_execute.create_task_latest(UNIVERSE_ID, PLACE_ID, engine_script, 300, false, plugin_content)
print("Waiting for task to finish")
local success = luau_execute.await_finish(task)
if not success then
local error = luau_execute.get_error(task)
assert(error, "could not fetch error from task")
stdio.ewrite("plugin upload failed because: task did not finish successfully\n\n")
stdio.ewrite(error.code)
stdio.ewrite("\n")
stdio.ewrite(error.message)
stdio.ewrite("\n")
process.exit(1)
end
print("Output from task:\n")
for _, log in luau_execute.get_structured_logs(task) do
if log.messageType == "ERROR" then
stdio.write(stdio.color("red"))
stdio.write(log.message)
stdio.write("\n")
stdio.write(stdio.color("reset"))
elseif log.messageType == "INFO" then
stdio.write(stdio.color("cyan"))
stdio.write(log.message)
stdio.write("\n")
stdio.write(stdio.color("reset"))
elseif log.messageType == "WARNING" then
stdio.write(stdio.color("yellow"))
stdio.write(log.message)
stdio.write("\n")
stdio.write(stdio.color("reset"))
else
stdio.write(stdio.color("reset"))
stdio.write(log.message)
stdio.write("\n")
stdio.write(stdio.color("reset"))
end
end

View File

@@ -30,11 +30,40 @@ Making a new release? Simply add the new header with the version and date undern
--> -->
## Unreleased ## Unreleased
* `inf` and `nan` values in properties are now synced ([#1176])
* Fixed a bug caused by having reference properties (such as `ObjectValue.Value`) that point to an Instance not included in syncback. ([#1179]) * Fixed a bug caused by having reference properties (such as `ObjectValue.Value`) that point to an Instance not included in syncback. ([#1179])
* Fixed instance replacement fallback failing when too many instances needed to be replaced. ([#1192]) * Fixed instance replacement fallback failing when too many instances needed to be replaced. ([#1192])
* Added actors and bindable/remote event/function variants to be synced back as JSON files. ([#1199])
* Fixed a bug where MacOS paths weren't being handled correctly. ([#1201])
* Fixed a bug where the notification timeout thread would fail to cancel on unmount ([#1211])
* Added a "Forget" option to the sync reminder notification to avoid being reminded for that place in the future ([#1215])
* Improves relative path calculation for sourcemap generation to avoid issues with Windows UNC paths. ([#1217])
* Fixed missing support for init.plugin.lua and init.plugin.luau. ([#1252])
* Add support for gitignore-style negation in `globIgnorePaths` and syncback's `ignorePaths` ([#1256])
* Fixed the sync fallback scrambling sibling order; replacements are now re-parented ancestors-first and in their original child order. ([#1265])
* Instances that share a name and class are now robustly matched on resync by comparing their properties, instead of relying on child order alone. ([#1266])
* Rojo now reports a clear error instead of panicking in several cases, including when the `serve` port is already in use, when a synced file is read-only or locked, when the filesystem watcher can't be created, and when the working directory is inaccessible. ([#1267])
* Fixed `/api/serialize` returning success when a requested instance ID is missing from the serve session tree. ([#1272])
* `rojo serve` now validates the `Host`/`Origin` headers to protect the local/private server against DNS rebinding, gates `/api/open` to local clients, and warns when bound to a network-reachable address. The accepted hosts can be extended with the `--allowed-hosts` option or a project's `serveAllowedHosts` field, for example to reach a network-exposed server by hostname. ([#1270])
* Fixed syncback not removing stale `$properties` entries when Studio resets a property to its engine default. ([#1244])
[#1176]: https://github.com/rojo-rbx/rojo/pull/1176
[#1179]: https://github.com/rojo-rbx/rojo/pull/1179 [#1179]: https://github.com/rojo-rbx/rojo/pull/1179
[#1192]: https://github.com/rojo-rbx/rojo/pull/1192 [#1192]: https://github.com/rojo-rbx/rojo/pull/1192
[#1199]: https://github.com/rojo-rbx/rojo/pull/1199
[#1201]: https://github.com/rojo-rbx/rojo/pull/1201
[#1211]: https://github.com/rojo-rbx/rojo/pull/1211
[#1215]: https://github.com/rojo-rbx/rojo/pull/1215
[#1217]: https://github.com/rojo-rbx/rojo/pull/1217
[#1252]: https://github.com/rojo-rbx/rojo/pull/1252
[#1256]: https://github.com/rojo-rbx/rojo/pull/1256
[#1265]: https://github.com/rojo-rbx/rojo/pull/1265
[#1266]: https://github.com/rojo-rbx/rojo/pull/1266
[#1267]: https://github.com/rojo-rbx/rojo/pull/1267
[#1272]: https://github.com/rojo-rbx/rojo/pull/1272
[#1270]: https://github.com/rojo-rbx/rojo/pull/1270
[#1244]: https://github.com/rojo-rbx/rojo/pull/1244
## [7.7.0-rc.1] (November 27th, 2025) ## [7.7.0-rc.1] (November 27th, 2025)

View File

@@ -13,11 +13,27 @@ Code contributions are welcome for features and bugs that have been reported in
You'll want these tools to work on Rojo: You'll want these tools to work on Rojo:
* Latest stable Rust compiler * Rust 1.88 or newer
* Latest stable [Rojo](https://github.com/rojo-rbx/rojo) * Rustfmt and Clippy are used for code formatting and linting.
* [Rokit](https://github.com/rojo-rbx/rokit) * [Rokit](https://github.com/rojo-rbx/rokit)
* [Luau Language Server](https://github.com/JohnnyMorganz/luau-lsp) (Only needed if working on the Studio plugin.) * [Luau Language Server](https://github.com/JohnnyMorganz/luau-lsp) (Only needed if working on the Studio plugin.)
Rokit installs the pinned Rojo, Selene, StyLua, Lune, and run-in-roblox versions listed in [`rokit.toml`](rokit.toml):
```bash
rokit install
```
Before opening a pull request, run the relevant checks:
```bash
cargo test
cargo fmt -- --check
cargo clippy
stylua --check plugin/src
selene plugin/src
```
When working on the Studio plugin, we recommend using this command to automatically rebuild the plugin when you save a change: When working on the Studio plugin, we recommend using this command to automatically rebuild the plugin when you save a change:
*(Make sure you've enabled the Studio setting to reload plugins on file change!)* *(Make sure you've enabled the Studio setting to reload plugins on file change!)*
@@ -28,7 +44,7 @@ bash scripts/watch-build-plugin.sh
You can also run the plugin's unit tests with the following: You can also run the plugin's unit tests with the following:
*(Make sure you have `run-in-roblox` installed first!)* *(If you are not using Rokit, make sure you have `run-in-roblox` installed first!)*
```bash ```bash
bash scripts/unit-test-plugin.sh bash scripts/unit-test-plugin.sh
@@ -48,26 +64,26 @@ Please file issues and we'll try to help figure out what the best way forward is
## Local Development Gotchas ## Local Development Gotchas
If your build fails with "Error: failed to open file `D:\code\rojo\plugin\modules\roact\src`" you need to update your Git submodules. If your build fails with an error about a missing path under `plugin/Packages`, such as `plugin/Packages/Roact`, you need to update your Git submodules.
Run the command and try building again: `git submodule update --init --recursive`. Run the command and try building again: `git submodule update --init --recursive`.
## Pushing a Rojo Release ## Pushing a Rojo Release
The Rojo release process is pretty manual right now. If you need to do it, here's how: The Rojo release process is driven by the GitHub Actions release workflow. If you need to do it, here's how:
1. Bump server version in [`Cargo.toml`](Cargo.toml) 1. Bump server version in [`Cargo.toml`](Cargo.toml)
2. Bump plugin version in [`plugin/src/Config.lua`](plugin/src/Config.lua) 2. Bump plugin version in [`plugin/Version.txt`](plugin/Version.txt)
3. Run `cargo test` to update `Cargo.lock` and run tests * The build checks that the Cargo and plugin versions match.
3. Run `cargo test` to update `Cargo.lock` after the version bump and run tests
4. Update [`CHANGELOG.md`](CHANGELOG.md) 4. Update [`CHANGELOG.md`](CHANGELOG.md)
5. Commit! 5. Commit!
* `git add . && git commit -m "Release vX.Y.Z"` * `git add . && git commit -m "Release vX.Y.Z"`
6. Tag the commit 6. Tag the commit
* `git tag vX.Y.Z` * `git tag vX.Y.Z`
7. Publish the CLI 7. Push commits and tags
* `cargo publish`
8. Publish the Plugin
* `cargo run -- upload plugin --asset_id 6415005344`
9. Push commits and tags
* `git push && git push --tags` * `git push && git push --tags`
8. Wait for the GitHub Actions release workflow to create the draft release and upload CLI/plugin artifacts
9. Publish the CLI crate
* `cargo publish`
10. Copy GitHub release content from previous release 10. Copy GitHub release content from previous release
* Update the leading text with a summary about the release * Update the leading text with a summary about the release
* Paste the changelog notes (as-is!) from [`CHANGELOG.md`](CHANGELOG.md) * Paste the changelog notes (as-is!) from [`CHANGELOG.md`](CHANGELOG.md)

20
Cargo.lock generated
View File

@@ -1319,6 +1319,7 @@ dependencies = [
"fs-err", "fs-err",
"notify", "notify",
"serde", "serde",
"tempfile",
] ]
[[package]] [[package]]
@@ -1519,6 +1520,12 @@ version = "1.0.15"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "57c0d7b74b563b49d38dae00a0c37d4d6de9b432382b2892f0574ddcae73fd0a" checksum = "57c0d7b74b563b49d38dae00a0c37d4d6de9b432382b2892f0574ddcae73fd0a"
[[package]]
name = "pathdiff"
version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df94ce210e5bc13cb6651479fa48d14f601d9858cfe0467f43ae157023b938d3"
[[package]] [[package]]
name = "percent-encoding" name = "percent-encoding"
version = "2.3.2" version = "2.3.2"
@@ -2067,6 +2074,7 @@ dependencies = [
"num_cpus", "num_cpus",
"opener", "opener",
"paste", "paste",
"pathdiff",
"pretty_assertions", "pretty_assertions",
"profiling", "profiling",
"rayon", "rayon",
@@ -2077,10 +2085,12 @@ dependencies = [
"rbx_xml", "rbx_xml",
"reqwest", "reqwest",
"ritz", "ritz",
"rmp-serde",
"roblox_install", "roblox_install",
"rojo-insta-ext", "rojo-insta-ext",
"semver", "semver",
"serde", "serde",
"serde_bytes",
"serde_json", "serde_json",
"serde_yaml", "serde_yaml",
"strum", "strum",
@@ -2221,6 +2231,16 @@ dependencies = [
"serde_derive", "serde_derive",
] ]
[[package]]
name = "serde_bytes"
version = "0.11.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a5d440709e79d88e51ac01c4b72fc6cb7314017bb7da9eeff678aa94c10e3ea8"
dependencies = [
"serde",
"serde_core",
]
[[package]] [[package]]
name = "serde_cbor" name = "serde_cbor"
version = "0.11.2" version = "0.11.2"

View File

@@ -100,10 +100,13 @@ clap = { version = "3.2.25", features = ["derive"] }
profiling = "1.0.15" profiling = "1.0.15"
yaml-rust2 = "0.10.3" yaml-rust2 = "0.10.3"
data-encoding = "2.8.0" data-encoding = "2.8.0"
pathdiff = "0.2.3"
blake3 = "1.5.0" blake3 = "1.5.0"
float-cmp = "0.9.0" float-cmp = "0.9.0"
indexmap = { version = "2.10.0", features = ["serde"] } indexmap = { version = "2.10.0", features = ["serde"] }
rmp-serde = "1.3.0"
serde_bytes = "0.11.19"
[target.'cfg(windows)'.dependencies] [target.'cfg(windows)'.dependencies]
winreg = "0.10.1" winreg = "0.10.1"
@@ -122,7 +125,7 @@ semver = "1.0.22"
rojo-insta-ext = { path = "crates/rojo-insta-ext" } rojo-insta-ext = { path = "crates/rojo-insta-ext" }
criterion = "0.3.6" criterion = "0.3.6"
insta = { version = "1.36.1", features = ["redactions", "yaml"] } insta = { version = "1.36.1", features = ["redactions", "yaml", "json"] }
paste = "1.0.14" paste = "1.0.14"
pretty_assertions = "1.4.0" pretty_assertions = "1.4.0"
serde_yaml = "0.8.26" serde_yaml = "0.8.26"

View File

@@ -25,12 +25,11 @@ Rojo enables:
* Versioning your game, library, or plugin using Git or another VCS * Versioning your game, library, or plugin using Git or another VCS
* Streaming `rbxmx` and `rbxm` models into your game in real time * Streaming `rbxmx` and `rbxm` models into your game in real time
* Packaging and deploying your project to Roblox.com from the command line * Packaging and deploying your project to Roblox.com from the command line
* Pulling Instances from Roblox place and model files back into an existing Rojo project with `rojo syncback`
In the future, Rojo will be able to: Rojo also has an optional two-way sync setting in the Studio plugin for syncing supported Studio edits back to the filesystem.
* Sync instances from Roblox Studio to the filesystem Some workflows, like fully automatic conversion of every existing game into a Rojo project, are still limited and may require manual project configuration.
* Automatically convert your existing game to work with Rojo
* Import custom instances like MoonScript code
## [Documentation](https://rojo.space/docs) ## [Documentation](https://rojo.space/docs)
Documentation is hosted in the [rojo.space repository](https://github.com/rojo-rbx/rojo.space). Documentation is hosted in the [rojo.space repository](https://github.com/rojo-rbx/rojo.space).

View File

@@ -30,6 +30,11 @@ fn snapshot_from_fs_path(path: &Path) -> io::Result<VfsSnapshot> {
continue; continue;
} }
// Ignore images in msgpack-luau because they aren't UTF-8 encoded.
if file_name.ends_with(".png") {
continue;
}
let child_snapshot = snapshot_from_fs_path(&entry.path())?; let child_snapshot = snapshot_from_fs_path(&entry.path())?;
children.push((file_name, child_snapshot)); children.push((file_name, child_snapshot));
} }
@@ -70,6 +75,7 @@ fn main() -> Result<(), anyhow::Error> {
"src" => snapshot_from_fs_path(&plugin_dir.join("src"))?, "src" => snapshot_from_fs_path(&plugin_dir.join("src"))?,
"Packages" => snapshot_from_fs_path(&plugin_dir.join("Packages"))?, "Packages" => snapshot_from_fs_path(&plugin_dir.join("Packages"))?,
"Version.txt" => snapshot_from_fs_path(&plugin_dir.join("Version.txt"))?, "Version.txt" => snapshot_from_fs_path(&plugin_dir.join("Version.txt"))?,
"UploadDetails.json" => snapshot_from_fs_path(&plugin_dir.join("UploadDetails.json"))?,
}), }),
}); });

View File

@@ -1,6 +1,10 @@
# memofs Changelog # memofs Changelog
## Unreleased Changes ## Unreleased Changes
* Added `Vfs::canonicalize`. [#1201]
* **Breaking:** `StdBackend::new` and `Vfs::new_default` now return `io::Result`, so a failure to create the filesystem watcher is reported as an error instead of panicking. The `Default` implementation for `StdBackend` has been removed as a result. [#1267]
[#1267]: https://github.com/rojo-rbx/rojo/pull/1267
## 0.3.1 (2025-11-27) ## 0.3.1 (2025-11-27)
* Added `Vfs::exists`. [#1169] * Added `Vfs::exists`. [#1169]

View File

@@ -19,3 +19,6 @@ crossbeam-channel = "0.5.12"
fs-err = "2.11.0" fs-err = "2.11.0"
notify = "4.0.17" notify = "4.0.17"
serde = { version = "1.0.197", features = ["derive"] } serde = { version = "1.0.197", features = ["derive"] }
[dev-dependencies]
tempfile = "3.10.1"

View File

@@ -232,6 +232,33 @@ impl VfsBackend for InMemoryFs {
} }
} }
// TODO: We rely on Rojo to prepend cwd to any relative path before storing paths
// in MemoFS. The current implementation will error if no prepended absolute path
// is found. It really only normalizes paths within the provided path's context.
// Example: "/Users/username/project/../other/file.txt" ->
// "/Users/username/other/file.txt"
// Erroneous example: "/Users/../../other/file.txt" -> "/other/file.txt"
// This is not very robust. We should implement proper path normalization here or otherwise
// warn if we are missing context and can not fully canonicalize the path correctly.
fn canonicalize(&mut self, path: &Path) -> io::Result<PathBuf> {
let mut normalized = PathBuf::new();
for component in path.components() {
match component {
std::path::Component::ParentDir => {
normalized.pop();
}
std::path::Component::CurDir => {}
_ => normalized.push(component),
}
}
let inner = self.inner.lock().unwrap();
match inner.entries.get(&normalized) {
Some(_) => Ok(normalized),
None => not_found(&normalized),
}
}
fn event_receiver(&self) -> crossbeam_channel::Receiver<VfsEvent> { fn event_receiver(&self) -> crossbeam_channel::Receiver<VfsEvent> {
let inner = self.inner.lock().unwrap(); let inner = self.inner.lock().unwrap();

View File

@@ -77,6 +77,7 @@ pub trait VfsBackend: sealed::Sealed + Send + 'static {
fn metadata(&mut self, path: &Path) -> io::Result<Metadata>; fn metadata(&mut self, path: &Path) -> io::Result<Metadata>;
fn remove_file(&mut self, path: &Path) -> io::Result<()>; fn remove_file(&mut self, path: &Path) -> io::Result<()>;
fn remove_dir_all(&mut self, path: &Path) -> io::Result<()>; fn remove_dir_all(&mut self, path: &Path) -> io::Result<()>;
fn canonicalize(&mut self, path: &Path) -> io::Result<PathBuf>;
fn event_receiver(&self) -> crossbeam_channel::Receiver<VfsEvent>; fn event_receiver(&self) -> crossbeam_channel::Receiver<VfsEvent>;
fn watch(&mut self, path: &Path) -> io::Result<()>; fn watch(&mut self, path: &Path) -> io::Result<()>;
@@ -225,6 +226,11 @@ impl VfsInner {
self.backend.metadata(path) self.backend.metadata(path)
} }
fn canonicalize<P: AsRef<Path>>(&mut self, path: P) -> io::Result<PathBuf> {
let path = path.as_ref();
self.backend.canonicalize(path)
}
fn event_receiver(&self) -> crossbeam_channel::Receiver<VfsEvent> { fn event_receiver(&self) -> crossbeam_channel::Receiver<VfsEvent> {
self.backend.event_receiver() self.backend.event_receiver()
} }
@@ -249,8 +255,11 @@ pub struct Vfs {
impl Vfs { impl Vfs {
/// Creates a new `Vfs` with the default backend, `StdBackend`. /// Creates a new `Vfs` with the default backend, `StdBackend`.
pub fn new_default() -> Self { ///
Self::new(StdBackend::new()) /// Returns an error if the filesystem watcher could not be initialized,
/// which can happen in restricted or sandboxed environments.
pub fn new_default() -> io::Result<Self> {
Ok(Self::new(StdBackend::new()?))
} }
/// Creates a new `Vfs` with the given backend. /// Creates a new `Vfs` with the given backend.
@@ -413,6 +422,19 @@ impl Vfs {
self.inner.lock().unwrap().metadata(path) self.inner.lock().unwrap().metadata(path)
} }
/// Normalize a path via the underlying backend.
///
/// Roughly equivalent to [`std::fs::canonicalize`][std::fs::canonicalize]. Relative paths are
/// resolved against the backend's current working directory (if applicable) and errors are
/// surfaced directly from the backend.
///
/// [std::fs::canonicalize]: https://doc.rust-lang.org/stable/std/fs/fn.canonicalize.html
#[inline]
pub fn canonicalize<P: AsRef<Path>>(&self, path: P) -> io::Result<PathBuf> {
let path = path.as_ref();
self.inner.lock().unwrap().canonicalize(path)
}
/// Retrieve a handle to the event receiver for this `Vfs`. /// Retrieve a handle to the event receiver for this `Vfs`.
#[inline] #[inline]
pub fn event_receiver(&self) -> crossbeam_channel::Receiver<VfsEvent> { pub fn event_receiver(&self) -> crossbeam_channel::Receiver<VfsEvent> {
@@ -540,6 +562,13 @@ impl VfsLock<'_> {
self.inner.metadata(path) self.inner.metadata(path)
} }
/// Normalize a path via the underlying backend.
#[inline]
pub fn normalize<P: AsRef<Path>>(&mut self, path: P) -> io::Result<PathBuf> {
let path = path.as_ref();
self.inner.canonicalize(path)
}
/// Retrieve a handle to the event receiver for this `Vfs`. /// Retrieve a handle to the event receiver for this `Vfs`.
#[inline] #[inline]
pub fn event_receiver(&self) -> crossbeam_channel::Receiver<VfsEvent> { pub fn event_receiver(&self) -> crossbeam_channel::Receiver<VfsEvent> {
@@ -555,7 +584,9 @@ impl VfsLock<'_> {
#[cfg(test)] #[cfg(test)]
mod test { mod test {
use crate::{InMemoryFs, Vfs, VfsSnapshot}; use crate::{InMemoryFs, StdBackend, Vfs, VfsSnapshot};
use std::io;
use std::path::PathBuf;
/// https://github.com/rojo-rbx/rojo/issues/899 /// https://github.com/rojo-rbx/rojo/issues/899
#[test] #[test]
@@ -571,4 +602,62 @@ mod test {
"bar\nfoo\n\n" "bar\nfoo\n\n"
); );
} }
/// https://github.com/rojo-rbx/rojo/issues/1200
#[test]
fn canonicalize_in_memory_success() {
let mut imfs = InMemoryFs::new();
let contents = "Lorem ipsum dolor sit amet.".to_string();
imfs.load_snapshot("/test/file.txt", VfsSnapshot::file(contents.to_string()))
.unwrap();
let vfs = Vfs::new(imfs);
assert_eq!(
vfs.canonicalize("/test/nested/../file.txt").unwrap(),
PathBuf::from("/test/file.txt")
);
assert_eq!(
vfs.read_to_string(vfs.canonicalize("/test/nested/../file.txt").unwrap())
.unwrap()
.to_string(),
contents.to_string()
);
}
#[test]
fn canonicalize_in_memory_missing_errors() {
let imfs = InMemoryFs::new();
let vfs = Vfs::new(imfs);
let err = vfs.canonicalize("test").unwrap_err();
assert_eq!(err.kind(), io::ErrorKind::NotFound);
}
#[test]
fn canonicalize_std_backend_success() {
let contents = "Lorem ipsum dolor sit amet.".to_string();
let dir = tempfile::tempdir().unwrap();
let file_path = dir.path().join("file.txt");
fs_err::write(&file_path, contents.to_string()).unwrap();
let vfs = Vfs::new(StdBackend::new().unwrap());
let canonicalized = vfs.canonicalize(&file_path).unwrap();
assert_eq!(canonicalized, file_path.canonicalize().unwrap());
assert_eq!(
vfs.read_to_string(&canonicalized).unwrap().to_string(),
contents.to_string()
);
}
#[test]
fn canonicalize_std_backend_missing_errors() {
let dir = tempfile::tempdir().unwrap();
let file_path = dir.path().join("test");
let vfs = Vfs::new(StdBackend::new().unwrap());
let err = vfs.canonicalize(&file_path).unwrap_err();
assert_eq!(err.kind(), io::ErrorKind::NotFound);
}
} }

View File

@@ -1,5 +1,5 @@
use std::io; use std::io;
use std::path::Path; use std::path::{Path, PathBuf};
use crate::{Metadata, ReadDir, VfsBackend, VfsEvent}; use crate::{Metadata, ReadDir, VfsBackend, VfsEvent};
@@ -50,6 +50,10 @@ impl VfsBackend for NoopBackend {
Err(io::Error::other("NoopBackend doesn't do anything")) Err(io::Error::other("NoopBackend doesn't do anything"))
} }
fn canonicalize(&mut self, _path: &Path) -> io::Result<PathBuf> {
Err(io::Error::other("NoopBackend doesn't do anything"))
}
fn event_receiver(&self) -> crossbeam_channel::Receiver<VfsEvent> { fn event_receiver(&self) -> crossbeam_channel::Receiver<VfsEvent> {
crossbeam_channel::never() crossbeam_channel::never()
} }

View File

@@ -17,9 +17,9 @@ pub struct StdBackend {
} }
impl StdBackend { impl StdBackend {
pub fn new() -> StdBackend { pub fn new() -> io::Result<StdBackend> {
let (notify_tx, notify_rx) = mpsc::channel(); let (notify_tx, notify_rx) = mpsc::channel();
let watcher = watcher(notify_tx, Duration::from_millis(50)).unwrap(); let watcher = watcher(notify_tx, Duration::from_millis(50)).map_err(io::Error::other)?;
let (tx, rx) = crossbeam_channel::unbounded(); let (tx, rx) = crossbeam_channel::unbounded();
@@ -46,11 +46,11 @@ impl StdBackend {
Result::<(), crossbeam_channel::SendError<VfsEvent>>::Ok(()) Result::<(), crossbeam_channel::SendError<VfsEvent>>::Ok(())
}); });
Self { Ok(Self {
watcher, watcher,
watcher_receiver: rx, watcher_receiver: rx,
watches: HashSet::new(), watches: HashSet::new(),
} })
} }
} }
@@ -106,6 +106,10 @@ impl VfsBackend for StdBackend {
}) })
} }
fn canonicalize(&mut self, path: &Path) -> io::Result<PathBuf> {
fs_err::canonicalize(path)
}
fn event_receiver(&self) -> crossbeam_channel::Receiver<VfsEvent> { fn event_receiver(&self) -> crossbeam_channel::Receiver<VfsEvent> {
self.watcher_receiver.clone() self.watcher_receiver.clone()
} }
@@ -130,9 +134,3 @@ impl VfsBackend for StdBackend {
self.watcher.unwatch(path).map_err(io::Error::other) self.watcher.unwatch(path).map_err(io::Error::other)
} }
} }
impl Default for StdBackend {
fn default() -> Self {
Self::new()
}
}

View File

@@ -22,6 +22,9 @@
}, },
"Version": { "Version": {
"$path": "plugin/Version.txt" "$path": "plugin/Version.txt"
},
"UploadDetails": {
"$path": "plugin/UploadDetails.json"
} }
} }
} }

View File

@@ -0,0 +1,7 @@
{
"assetId": 13916111004,
"name": "Rojo",
"description": "The plugin portion of Rojo, a tool to enable professional tooling for Roblox developers.",
"creatorId": 32644114,
"creatorType": "Group"
}

View File

@@ -1,5 +1,7 @@
local HttpService = game:GetService("HttpService") local HttpService = game:GetService("HttpService")
local msgpack = require(script.Parent.Parent.msgpack)
local stringTemplate = [[ local stringTemplate = [[
Http.Response { Http.Response {
code: %d code: %d
@@ -31,4 +33,8 @@ function Response:json()
return HttpService:JSONDecode(self.body) return HttpService:JSONDecode(self.body)
end end
function Response:msgpack()
return msgpack.decode(self.body)
end
return Response return Response

View File

@@ -1,7 +1,8 @@
local HttpService = game:GetService("HttpService") local HttpService = game:GetService("HttpService")
local Promise = require(script.Parent.Promise)
local Log = require(script.Parent.Log) local Log = require(script.Parent.Log)
local msgpack = require(script.Parent.msgpack)
local Promise = require(script.Parent.Promise)
local HttpError = require(script.Error) local HttpError = require(script.Error)
local HttpResponse = require(script.Response) local HttpResponse = require(script.Response)
@@ -13,6 +14,13 @@ local Http = {}
Http.Error = HttpError Http.Error = HttpError
Http.Response = HttpResponse Http.Response = HttpResponse
-- Monkey patch msgpack.UInt64.new to lossily convert the low and high bits of the integer
-- to a native Luau number. We should change the upstream decoder to emit a native
-- integer, once those are live.
function msgpack.UInt64.new(mostSignificantPart: number, leastSignificantPart: number): number
return (mostSignificantPart % 2 ^ 32) * 2 ^ 32 + (leastSignificantPart % 2 ^ 32)
end
local function performRequest(requestParams) local function performRequest(requestParams)
local requestId = lastRequestId + 1 local requestId = lastRequestId + 1
lastRequestId = requestId lastRequestId = requestId
@@ -68,4 +76,12 @@ function Http.jsonDecode(source)
return HttpService:JSONDecode(source) return HttpService:JSONDecode(source)
end end
function Http.msgpackEncode(object)
return msgpack.encode(object)
end
function Http.msgpackDecode(source)
return msgpack.decode(source)
end
return Http return Http

View File

@@ -1,8 +1,8 @@
local ReplicatedStorage = game:GetService("ReplicatedStorage") local ReplicatedStorage = game:GetService("ReplicatedStorage")
local TestEZ = require(ReplicatedStorage.Packages:WaitForChild("TestEZ", 10)) local TestEZ = require(ReplicatedStorage:WaitForChild("Packages", 10):WaitForChild("TestEZ", 10))
local Rojo = ReplicatedStorage.Rojo local Rojo = ReplicatedStorage:WaitForChild("Rojo", 10)
local Settings = require(Rojo.Plugin.Settings) local Settings = require(Rojo.Plugin.Settings)
Settings:set("logLevel", "Trace") Settings:set("logLevel", "Trace")

View File

@@ -145,7 +145,7 @@ function ApiContext:connect()
return Http.get(url) return Http.get(url)
:andThen(rejectFailedRequests) :andThen(rejectFailedRequests)
:andThen(Http.Response.json) :andThen(Http.Response.msgpack)
:andThen(rejectWrongProtocolVersion) :andThen(rejectWrongProtocolVersion)
:andThen(function(body) :andThen(function(body)
assert(validateApiInfo(body)) assert(validateApiInfo(body))
@@ -163,7 +163,7 @@ end
function ApiContext:read(ids) function ApiContext:read(ids)
local url = ("%s/api/read/%s"):format(self.__baseUrl, table.concat(ids, ",")) local url = ("%s/api/read/%s"):format(self.__baseUrl, table.concat(ids, ","))
return Http.get(url):andThen(rejectFailedRequests):andThen(Http.Response.json):andThen(function(body) return Http.get(url):andThen(rejectFailedRequests):andThen(Http.Response.msgpack):andThen(function(body)
if body.sessionId ~= self.__sessionId then if body.sessionId ~= self.__sessionId then
return Promise.reject("Server changed ID") return Promise.reject("Server changed ID")
end end
@@ -191,9 +191,9 @@ function ApiContext:write(patch)
table.insert(updated, fixedUpdate) table.insert(updated, fixedUpdate)
end end
-- Only add the 'added' field if the table is non-empty, or else Roblox's -- Only add the 'added' field if the table is non-empty, or else the msgpack
-- JSON implementation will turn the table into an array instead of an -- encode implementation will turn the table into an array instead of a map,
-- object, causing API validation to fail. -- causing API validation to fail.
local added local added
if next(patch.added) ~= nil then if next(patch.added) ~= nil then
added = patch.added added = patch.added
@@ -206,13 +206,16 @@ function ApiContext:write(patch)
added = added, added = added,
} }
body = Http.jsonEncode(body) body = Http.msgpackEncode(body)
return Http.post(url, body):andThen(rejectFailedRequests):andThen(Http.Response.json):andThen(function(responseBody) return Http.post(url, body)
Log.info("Write response: {:?}", responseBody) :andThen(rejectFailedRequests)
:andThen(Http.Response.msgpack)
:andThen(function(responseBody)
Log.info("Write response: {:?}", responseBody)
return responseBody return responseBody
end) end)
end end
function ApiContext:connectWebSocket(packetHandlers) function ApiContext:connectWebSocket(packetHandlers)
@@ -234,7 +237,7 @@ function ApiContext:connectWebSocket(packetHandlers)
local closed, errored, received local closed, errored, received
received = self.__wsClient.MessageReceived:Connect(function(msg) received = self.__wsClient.MessageReceived:Connect(function(msg)
local data = Http.jsonDecode(msg) local data = Http.msgpackDecode(msg)
if data.sessionId ~= self.__sessionId then if data.sessionId ~= self.__sessionId then
Log.warn("Received message with wrong session ID; ignoring") Log.warn("Received message with wrong session ID; ignoring")
return return
@@ -280,7 +283,7 @@ end
function ApiContext:open(id) function ApiContext:open(id)
local url = ("%s/api/open/%s"):format(self.__baseUrl, id) local url = ("%s/api/open/%s"):format(self.__baseUrl, id)
return Http.post(url, ""):andThen(rejectFailedRequests):andThen(Http.Response.json):andThen(function(body) return Http.post(url, ""):andThen(rejectFailedRequests):andThen(Http.Response.msgpack):andThen(function(body)
if body.sessionId ~= self.__sessionId then if body.sessionId ~= self.__sessionId then
return Promise.reject("Server changed ID") return Promise.reject("Server changed ID")
end end
@@ -291,11 +294,11 @@ end
function ApiContext:serialize(ids: { string }) function ApiContext:serialize(ids: { string })
local url = ("%s/api/serialize"):format(self.__baseUrl) local url = ("%s/api/serialize"):format(self.__baseUrl)
local request_body = Http.jsonEncode({ sessionId = self.__sessionId, ids = ids }) local request_body = Http.msgpackEncode({ sessionId = self.__sessionId, ids = ids })
return Http.post(url, request_body) return Http.post(url, request_body)
:andThen(rejectFailedRequests) :andThen(rejectFailedRequests)
:andThen(Http.Response.json) :andThen(Http.Response.msgpack)
:andThen(function(response_body) :andThen(function(response_body)
if response_body.sessionId ~= self.__sessionId then if response_body.sessionId ~= self.__sessionId then
return Promise.reject("Server changed ID") return Promise.reject("Server changed ID")
@@ -309,11 +312,11 @@ end
function ApiContext:refPatch(ids: { string }) function ApiContext:refPatch(ids: { string })
local url = ("%s/api/ref-patch"):format(self.__baseUrl) local url = ("%s/api/ref-patch"):format(self.__baseUrl)
local request_body = Http.jsonEncode({ sessionId = self.__sessionId, ids = ids }) local request_body = Http.msgpackEncode({ sessionId = self.__sessionId, ids = ids })
return Http.post(url, request_body) return Http.post(url, request_body)
:andThen(rejectFailedRequests) :andThen(rejectFailedRequests)
:andThen(Http.Response.json) :andThen(Http.Response.msgpack)
:andThen(function(response_body) :andThen(function(response_body)
if response_body.sessionId ~= self.__sessionId then if response_body.sessionId ~= self.__sessionId then
return Promise.reject("Server changed ID") return Promise.reject("Server changed ID")

View File

@@ -19,9 +19,15 @@ local FullscreenNotification = Roact.Component:extend("FullscreeFullscreenNotifi
function FullscreenNotification:init() function FullscreenNotification:init()
self.transparency, self.setTransparency = Roact.createBinding(0) self.transparency, self.setTransparency = Roact.createBinding(0)
self.lifetime = self.props.timeout self.lifetime = self.props.timeout
self.dismissed = false
end end
function FullscreenNotification:dismiss() function FullscreenNotification:dismiss()
if self.dismissed then
return
end
self.dismissed = true
if self.props.onClose then if self.props.onClose then
self.props.onClose() self.props.onClose()
end end
@@ -59,7 +65,7 @@ function FullscreenNotification:didMount()
end end
function FullscreenNotification:willUnmount() function FullscreenNotification:willUnmount()
if self.timeout and coroutine.status(self.timeout) ~= "dead" then if self.timeout and coroutine.status(self.timeout) == "suspended" then
task.cancel(self.timeout) task.cancel(self.timeout)
end end
end end

View File

@@ -25,6 +25,7 @@ function Notification:init()
self.binding = bindingUtil.fromMotor(self.motor) self.binding = bindingUtil.fromMotor(self.motor)
self.lifetime = self.props.timeout self.lifetime = self.props.timeout
self.dismissed = false
self.motor:onStep(function(value) self.motor:onStep(function(value)
if value <= 0 and self.props.onClose then if value <= 0 and self.props.onClose then
@@ -34,6 +35,11 @@ function Notification:init()
end end
function Notification:dismiss() function Notification:dismiss()
if self.dismissed then
return
end
self.dismissed = true
self.motor:setGoal(Flipper.Spring.new(0, { self.motor:setGoal(Flipper.Spring.new(0, {
frequency = 5, frequency = 5,
dampingRatio = 1, dampingRatio = 1,
@@ -75,7 +81,7 @@ function Notification:didMount()
end end
function Notification:willUnmount() function Notification:willUnmount()
if self.timeout and coroutine.status(self.timeout) ~= "dead" then if self.timeout and coroutine.status(self.timeout) == "suspended" then
task.cancel(self.timeout) task.cancel(self.timeout)
end end
end end

View File

@@ -301,6 +301,19 @@ function App:setPriorSyncInfo(host: string, port: string, projectName: string)
Settings:set("priorEndpoints", priorSyncInfos) Settings:set("priorEndpoints", priorSyncInfos)
end end
function App:forgetPriorSyncInfo()
local priorSyncInfos = Settings:get("priorEndpoints")
if not priorSyncInfos then
priorSyncInfos = {}
end
local id = tostring(game.PlaceId)
priorSyncInfos[id] = nil
Log.trace("Erased last used endpoint for {}", game.PlaceId)
Settings:set("priorEndpoints", priorSyncInfos)
end
function App:getHostAndPort() function App:getHostAndPort()
local host = self.host:getValue() local host = self.host:getValue()
local port = self.port:getValue() local port = self.port:getValue()
@@ -435,7 +448,8 @@ function App:checkSyncReminder()
self:findActiveServer() self:findActiveServer()
:andThen(function(serverInfo, host, port) :andThen(function(serverInfo, host, port)
self:sendSyncReminder( self:sendSyncReminder(
`Project '{serverInfo.projectName}' is serving at {host}:{port}.\nWould you like to connect?` `Project '{serverInfo.projectName}' is serving at {host}:{port}.\nWould you like to connect?`,
{ "Connect", "Dismiss" }
) )
end) end)
:catch(function() :catch(function()
@@ -446,7 +460,8 @@ function App:checkSyncReminder()
local timeSinceSync = timeUtil.elapsedToText(os.time() - priorSyncInfo.timestamp) local timeSinceSync = timeUtil.elapsedToText(os.time() - priorSyncInfo.timestamp)
self:sendSyncReminder( self:sendSyncReminder(
`You synced project '{priorSyncInfo.projectName}' to this place {timeSinceSync}.\nDid you mean to run 'rojo serve' and then connect?` `You synced project '{priorSyncInfo.projectName}' to this place {timeSinceSync}.\nDid you mean to run 'rojo serve' and then connect?`,
{ "Connect", "Forget", "Dismiss" }
) )
end end
end) end)
@@ -486,12 +501,16 @@ function App:stopSyncReminderPolling()
end end
end end
function App:sendSyncReminder(message: string) function App:sendSyncReminder(message: string, shownActions: { string })
local syncReminderMode = Settings:get("syncReminderMode") local syncReminderMode = Settings:get("syncReminderMode")
if syncReminderMode == "None" then if syncReminderMode == "None" then
return return
end end
local connectIndex = table.find(shownActions, "Connect")
local forgetIndex = table.find(shownActions, "Forget")
local dismissIndex = table.find(shownActions, "Dismiss")
self.dismissSyncReminder = self:addNotification({ self.dismissSyncReminder = self:addNotification({
text = message, text = message,
timeout = 120, timeout = 120,
@@ -500,24 +519,39 @@ function App:sendSyncReminder(message: string)
self.dismissSyncReminder = nil self.dismissSyncReminder = nil
end, end,
actions = { actions = {
Connect = { Connect = if connectIndex
text = "Connect", then {
style = "Solid", text = "Connect",
layoutOrder = 1, style = "Solid",
onClick = function() layoutOrder = connectIndex,
self:startSession() onClick = function()
end, self:startSession()
}, end,
Dismiss = { }
text = "Dismiss", else nil,
style = "Bordered", Forget = if forgetIndex
layoutOrder = 2, then {
onClick = function() text = "Forget",
-- If the user dismisses the reminder, style = "Bordered",
-- then we don't need to remind them again layoutOrder = forgetIndex,
self:stopSyncReminderPolling() onClick = function()
end, -- The user doesn't want to be reminded again about this sync
}, self:forgetPriorSyncInfo()
end,
}
else nil,
Dismiss = if dismissIndex
then {
text = "Dismiss",
style = "Bordered",
layoutOrder = dismissIndex,
onClick = function()
-- If the user dismisses the reminder,
-- then we don't need to remind them again
self:stopSyncReminderPolling()
end,
}
else nil,
}, },
}) })
end end

View File

@@ -0,0 +1,44 @@
--[[
Counts how many of a virtual instance's properties match the live values on a
candidate Roblox instance. `hydrate` uses this to break ties when several
existing children share the same Name and ClassName.
This mirrors the read -> decode -> compare flow that `diff` uses, reusing the
same `getProperty`, `decodeValue`, and `trueEquals` helpers.
]]
local getProperty = require(script.Parent.getProperty)
local decodeValue = require(script.Parent.decodeValue)
local trueEquals = require(script.Parent.trueEquals)
local function countMatchingProperties(instance, virtualInstance, instanceMap)
local score = 0
for propertyName, virtualValue in virtualInstance.Properties do
-- Skip refs. During hydration the instanceMap is still being built
-- top-down, so a ref may point at an instance we haven't hydrated yet
-- and therefore can't decode reliably. Refs are also a poor
-- disambiguator between same-named siblings.
if next(virtualValue) == "Ref" then
continue
end
local getSuccess, existingValue = getProperty(instance, propertyName)
if not getSuccess then
continue
end
local decodeSuccess, decodedValue = decodeValue(virtualValue, instanceMap)
if not decodeSuccess then
continue
end
if trueEquals(existingValue, decodedValue) then
score += 1
end
end
return score
end
return countMatchingProperties

View File

@@ -0,0 +1,91 @@
return function()
local countMatchingProperties = require(script.Parent.countMatchingProperties)
local InstanceMap = require(script.Parent.Parent.InstanceMap)
it("counts properties whose values match the instance", function()
local instance = Instance.new("StringValue")
instance.Value = "hello"
local virtualInstance = {
ClassName = "StringValue",
Name = "Value",
Properties = {
Value = { String = "hello" },
},
Children = {},
}
expect(countMatchingProperties(instance, virtualInstance, InstanceMap.new())).to.equal(1)
end)
it("does not count properties whose values differ", function()
local instance = Instance.new("StringValue")
instance.Value = "hello"
local virtualInstance = {
ClassName = "StringValue",
Name = "Value",
Properties = {
Value = { String = "different" },
},
Children = {},
}
expect(countMatchingProperties(instance, virtualInstance, InstanceMap.new())).to.equal(0)
end)
it("counts multiple matching properties independently", function()
local instance = Instance.new("Part")
instance.Anchored = true
instance.CanCollide = false
local virtualInstance = {
ClassName = "Part",
Name = "Part",
Properties = {
Anchored = { Bool = true },
CanCollide = { Bool = false },
},
Children = {},
}
expect(countMatchingProperties(instance, virtualInstance, InstanceMap.new())).to.equal(2)
-- Flip one so only a single property matches.
instance.CanCollide = true
expect(countMatchingProperties(instance, virtualInstance, InstanceMap.new())).to.equal(1)
end)
it("skips unknown properties without counting or erroring", function()
local instance = Instance.new("Folder")
local virtualInstance = {
ClassName = "Folder",
Name = "Folder",
Properties = {
FAKE_PROPERTY = { String = "nope" },
},
Children = {},
}
expect(countMatchingProperties(instance, virtualInstance, InstanceMap.new())).to.equal(0)
end)
it("skips Ref properties without counting or erroring", function()
local instance = Instance.new("ObjectValue")
local virtualInstance = {
ClassName = "ObjectValue",
Name = "ObjectValue",
Properties = {
-- A ref must be skipped rather than decoded: during hydration
-- the target may not be in the map yet.
Value = { Ref = "00000000000000000000000000000000" },
},
Children = {},
}
expect(countMatchingProperties(instance, virtualInstance, InstanceMap.new())).to.equal(0)
end)
end

View File

@@ -10,100 +10,12 @@ local invariant = require(script.Parent.Parent.invariant)
local getProperty = require(script.Parent.getProperty) local getProperty = require(script.Parent.getProperty)
local Error = require(script.Parent.Error) local Error = require(script.Parent.Error)
local decodeValue = require(script.Parent.decodeValue) local decodeValue = require(script.Parent.decodeValue)
local trueEquals = require(script.Parent.trueEquals)
local function isEmpty(table) local function isEmpty(table)
return next(table) == nil return next(table) == nil
end end
local function fuzzyEq(a: number, b: number, epsilon: number): boolean
return math.abs(a - b) < epsilon
end
local function trueEquals(a, b): boolean
-- Exit early for simple equality values
if a == b then
return true
end
-- Treat nil and { Ref = "000...0" } as equal
if
(a == nil and type(b) == "table" and b.Ref == "00000000000000000000000000000000")
or (b == nil and type(a) == "table" and a.Ref == "00000000000000000000000000000000")
then
return true
end
local typeA, typeB = typeof(a), typeof(b)
-- For tables, try recursive deep equality
if typeA == "table" and typeB == "table" then
local checkedKeys = {}
for key, value in a do
checkedKeys[key] = true
if not trueEquals(value, b[key]) then
return false
end
end
for key, value in b do
if checkedKeys[key] then
continue
end
if not trueEquals(value, a[key]) then
return false
end
end
return true
-- For numbers, compare with epsilon of 0.0001 to avoid floating point inequality
elseif typeA == "number" and typeB == "number" then
return fuzzyEq(a, b, 0.0001)
-- For EnumItem->number, compare the EnumItem's value
elseif typeA == "number" and typeB == "EnumItem" then
return a == b.Value
elseif typeA == "EnumItem" and typeB == "number" then
return a.Value == b
-- For Color3s, compare to RGB ints to avoid floating point inequality
elseif typeA == "Color3" and typeB == "Color3" then
local aR, aG, aB = math.floor(a.R * 255), math.floor(a.G * 255), math.floor(a.B * 255)
local bR, bG, bB = math.floor(b.R * 255), math.floor(b.G * 255), math.floor(b.B * 255)
return aR == bR and aG == bG and aB == bB
-- For CFrames, compare to components with epsilon of 0.0001 to avoid floating point inequality
elseif typeA == "CFrame" and typeB == "CFrame" then
local aComponents, bComponents = { a:GetComponents() }, { b:GetComponents() }
for i, aComponent in aComponents do
if not fuzzyEq(aComponent, bComponents[i], 0.0001) then
return false
end
end
return true
-- For Vector3s, compare to components with epsilon of 0.0001 to avoid floating point inequality
elseif typeA == "Vector3" and typeB == "Vector3" then
local aComponents, bComponents = { a.X, a.Y, a.Z }, { b.X, b.Y, b.Z }
for i, aComponent in aComponents do
if not fuzzyEq(aComponent, bComponents[i], 0.0001) then
return false
end
end
return true
-- For Vector2s, compare to components with epsilon of 0.0001 to avoid floating point inequality
elseif typeA == "Vector2" and typeB == "Vector2" then
local aComponents, bComponents = { a.X, a.Y }, { b.X, b.Y }
for i, aComponent in aComponents do
if not fuzzyEq(aComponent, bComponents[i], 0.0001) then
return false
end
end
return true
end
return false
end
local function shouldDeleteUnknownInstances(virtualInstance) local function shouldDeleteUnknownInstances(virtualInstance)
if virtualInstance.Metadata ~= nil then if virtualInstance.Metadata ~= nil then
return not virtualInstance.Metadata.ignoreUnknownInstances return not virtualInstance.Metadata.ignoreUnknownInstances

View File

@@ -3,9 +3,22 @@
concrete instances and assigning them IDs. concrete instances and assigning them IDs.
]] ]]
local invariant = require(script.Parent.Parent.invariant) local Packages = script.Parent.Parent.Parent.Packages
local Log = require(Packages.Log)
local function hydrate(instanceMap, virtualInstances, rootId, rootInstance) local invariant = require(script.Parent.Parent.invariant)
local countMatchingProperties = require(script.Parent.countMatchingProperties)
-- When several existing children share a Name and ClassName we disambiguate
-- them by scoring how well each one's properties match the virtual instance.
-- That scoring is far more expensive than a Name/ClassName check, so we only do
-- it for reasonably-sized groups. Larger groups (e.g. a folder with thousands of
-- identically-named parts) fall back to the original order-based matching, which
-- bounds the added work to roughly MAX_CANDIDATES_TO_SCORE^2 property reads per
-- group regardless of how large the group is.
local MAX_CANDIDATES_TO_SCORE = 32
local function hydrateInner(stats, instanceMap, virtualInstances, rootId, rootInstance)
local virtualInstance = virtualInstances[rootId] local virtualInstance = virtualInstances[rootId]
if virtualInstance == nil then if virtualInstance == nil then
@@ -13,38 +26,163 @@ local function hydrate(instanceMap, virtualInstances, rootId, rootInstance)
end end
instanceMap:insert(rootId, rootInstance) instanceMap:insert(rootId, rootInstance)
stats.hydrated += 1
local existingChildren = rootInstance:GetChildren() local existingChildren = rootInstance:GetChildren()
-- For each existing child, we'll track whether it's been paired with an -- Group existing children by Name then ClassName so each virtual child can
-- instance that the Rojo server knows about. -- find its candidate matches without scanning every sibling. This is what
local isExistingChildVisited = {} -- keeps hydration fast for parents with thousands of children. Nesting the
for i = 1, #existingChildren do -- two tables (rather than a combined key) keeps the Name and ClassName checks
isExistingChildVisited[i] = false -- exact, with no way for one to bleed into the other.
local buckets = {}
for _, childInstance in existingChildren do
-- We guard accessing Name and ClassName in order to avoid tripping over
-- children of DataModel that Rojo won't have permissions to access at all.
local accessSuccess, name, className = pcall(function()
return childInstance.Name, childInstance.ClassName
end)
if not accessSuccess then
continue
end
local bucketsByClassName = buckets[name]
if bucketsByClassName == nil then
bucketsByClassName = {}
buckets[name] = bucketsByClassName
end
local bucket = bucketsByClassName[className]
if bucket == nil then
bucket = { cursor = 1, instances = {} }
bucketsByClassName[className] = bucket
end
table.insert(bucket.instances, childInstance)
end end
-- Tracks which existing children have already been paired, so one instance
-- isn't matched to two different virtual instances.
local visited = {}
for _, childId in ipairs(virtualInstance.Children) do for _, childId in ipairs(virtualInstance.Children) do
local virtualChild = virtualInstances[childId] local virtualChild = virtualInstances[childId]
for childIndex, childInstance in existingChildren do local bucketsByClassName = buckets[virtualChild.Name]
if not isExistingChildVisited[childIndex] then local bucket = bucketsByClassName and bucketsByClassName[virtualChild.ClassName]
-- We guard accessing Name and ClassName in order to avoid if bucket == nil then
-- tripping over children of DataModel that Rojo won't have -- No existing instance matches; diff will mark this id for creation.
-- permissions to access at all. Log.trace(
local accessSuccess, name, className = pcall(function() "hydrate: no existing instance matches {} ({}) for id {}",
return childInstance.Name, childInstance.ClassName virtualChild.Name,
end) virtualChild.ClassName,
childId
)
continue
end
-- This rule is very conservative and could be loosened in the local instances = bucket.instances
-- future, or more heuristics could be introduced.
if accessSuccess and name == virtualChild.Name and className == virtualChild.ClassName then -- Advance past any leading children that have already been paired. The
isExistingChildVisited[childIndex] = true -- cursor makes order-based matching amortized O(1) per child even for
hydrate(instanceMap, virtualInstances, childId, childInstance) -- very large groups, rather than rescanning the visited prefix.
break while bucket.cursor <= #instances and visited[instances[bucket.cursor]] do
bucket.cursor += 1
end
if bucket.cursor > #instances then
-- Every matching instance has already been paired with an earlier id.
Log.trace(
"hydrate: no unpaired instance left for {} ({}) for id {}",
virtualChild.Name,
virtualChild.ClassName,
childId
)
continue
end
-- The cursor points at the earliest unvisited child, so the slots from
-- here to the end bound how many candidates remain. Visited children
-- after the cursor (gaps) only appear once a group is small enough to be
-- scored -- the order-based path below always takes the earliest, which
-- keeps the visited region a contiguous prefix. So whenever this count
-- exceeds the cap it is exact, and we can pick the earliest match without
-- collecting anything.
local remaining = #instances - bucket.cursor + 1
local match
if remaining > MAX_CANDIDATES_TO_SCORE then
-- Too many to score affordably; take the earliest in child order,
-- reproducing the original Name + ClassName behavior.
match = instances[bucket.cursor]
Log.trace(
"hydrate: {} candidates named {} ({}) exceeds the scoring cap of {}; matching id {} by child order",
remaining,
virtualChild.Name,
virtualChild.ClassName,
MAX_CANDIDATES_TO_SCORE,
childId
)
else
-- Collect the (at most `remaining`) unvisited candidates.
local candidates = {}
for index = bucket.cursor, #instances do
local childInstance = instances[index]
if not visited[childInstance] then
table.insert(candidates, childInstance)
end end
end end
if #candidates == 1 then
-- Only one candidate, so there's nothing to disambiguate.
match = candidates[1]
else
-- Break the tie by choosing the candidate whose properties best
-- match the virtual instance, falling back to the earliest in
-- child order when scores are equal.
local bestScore = -1
for _, childInstance in candidates do
local score = countMatchingProperties(childInstance, virtualChild, instanceMap)
if score > bestScore then
bestScore = score
match = childInstance
end
end
stats.ambiguousGroups += 1
stats.candidatesScored += #candidates
Log.trace(
"hydrate: disambiguated {} candidates named {} ({}) for id {} by property match (best score {})",
#candidates,
virtualChild.Name,
virtualChild.ClassName,
childId,
bestScore
)
end
end end
visited[match] = true
hydrateInner(stats, instanceMap, virtualInstances, childId, match)
end end
end end
local function hydrate(instanceMap, virtualInstances, rootId, rootInstance)
-- Tallies of the work hydration did, surfaced in a single debug log below so
-- the cost of property-based disambiguation is visible without per-node spam.
local stats = {
hydrated = 0,
ambiguousGroups = 0,
candidatesScored = 0,
}
hydrateInner(stats, instanceMap, virtualInstances, rootId, rootInstance)
Log.debug(
"Hydrated {} instances ({} ambiguous name+class groups, {} candidates scored)",
stats.hydrated,
stats.ambiguousGroups,
stats.candidatesScored
)
end
return hydrate return hydrate

View File

@@ -126,4 +126,140 @@ return function()
expect(knownInstances.fromIds["CHILD1"]).to.equal(child1) expect(knownInstances.fromIds["CHILD1"]).to.equal(child1)
expect(knownInstances.fromIds["CHILD2"]).to.equal(child2) expect(knownInstances.fromIds["CHILD2"]).to.equal(child2)
end) end)
it("should disambiguate duplicate-named siblings by matching properties", function()
local knownInstances = InstanceMap.new()
local virtualInstances = {
ROOT = {
ClassName = "Folder",
Name = "Root",
Properties = {},
Children = { "CHILD_A", "CHILD_B" },
},
CHILD_A = {
ClassName = "StringValue",
Name = "a",
Properties = { Value = { String = "first" } },
Children = {},
},
CHILD_B = {
ClassName = "StringValue",
Name = "a",
Properties = { Value = { String = "second" } },
Children = {},
},
}
local rootInstance = Instance.new("Folder")
-- Created in the reverse order of the virtual children, so a purely
-- order-based tiebreak would mis-pair them.
local child1 = Instance.new("StringValue")
child1.Name = "a"
child1.Value = "second"
child1.Parent = rootInstance
local child2 = Instance.new("StringValue")
child2.Name = "a"
child2.Value = "first"
child2.Parent = rootInstance
hydrate(knownInstances, virtualInstances, "ROOT", rootInstance)
expect(knownInstances:size()).to.equal(3)
expect(knownInstances.fromIds["CHILD_A"]).to.equal(child2)
expect(knownInstances.fromIds["CHILD_B"]).to.equal(child1)
end)
it("should fall back to child order for duplicate-named siblings with no distinguishing properties", function()
local knownInstances = InstanceMap.new()
local virtualInstances = {
ROOT = {
ClassName = "Folder",
Name = "Root",
Properties = {},
Children = { "CHILD_A", "CHILD_B" },
},
CHILD_A = {
ClassName = "Folder",
Name = "a",
Properties = {},
Children = {},
},
CHILD_B = {
ClassName = "Folder",
Name = "a",
Properties = {},
Children = {},
},
}
local rootInstance = Instance.new("Folder")
local child1 = Instance.new("Folder")
child1.Name = "a"
child1.Parent = rootInstance
local child2 = Instance.new("Folder")
child2.Name = "a"
child2.Parent = rootInstance
hydrate(knownInstances, virtualInstances, "ROOT", rootInstance)
expect(knownInstances:size()).to.equal(3)
-- With equal scores the earliest unvisited child wins, preserving the
-- original order-based behavior.
expect(knownInstances.fromIds["CHILD_A"]).to.equal(child1)
expect(knownInstances.fromIds["CHILD_B"]).to.equal(child2)
end)
it("should fall back to child order for very large duplicate-named groups", function()
-- More candidates than hydrate is willing to score at once. The group
-- must fall back to order-based matching, so virtual child N pairs with
-- existing child N regardless of properties.
local count = 64
local knownInstances = InstanceMap.new()
local virtualInstances = {
ROOT = {
ClassName = "Folder",
Name = "Root",
Properties = {},
Children = {},
},
}
local rootInstance = Instance.new("Folder")
local expectedInstances = {}
for i = 1, count do
local id = "CHILD_" .. i
table.insert(virtualInstances.ROOT.Children, id)
virtualInstances[id] = {
ClassName = "StringValue",
Name = "a",
-- Distinct values that, if scored, would pair by value rather
-- than by order.
Properties = { Value = { String = "value " .. i } },
Children = {},
}
local child = Instance.new("StringValue")
child.Name = "a"
child.Value = "value " .. (count - i + 1)
child.Parent = rootInstance
expectedInstances[id] = child
end
hydrate(knownInstances, virtualInstances, "ROOT", rootInstance)
expect(knownInstances:size()).to.equal(count + 1)
for id, expectedInstance in expectedInstances do
expect(knownInstances.fromIds[id]).to.equal(expectedInstance)
end
end)
end end

View File

@@ -0,0 +1,100 @@
--[[
Fuzzy value-equality used to compare a decoded virtual property value against
the live value read from a real instance. Shared by `diff` (to decide whether
a property changed) and `hydrate` (to score candidate instances).
]]
local function fuzzyEq(a: number, b: number, epsilon: number): boolean
return math.abs(a - b) < epsilon
end
local function trueEquals(a, b): boolean
-- Exit early for simple equality values
if a == b then
return true
end
-- Treat nil and { Ref = "000...0" } as equal
if
(a == nil and type(b) == "table" and b.Ref == "00000000000000000000000000000000")
or (b == nil and type(a) == "table" and a.Ref == "00000000000000000000000000000000")
then
return true
end
local typeA, typeB = typeof(a), typeof(b)
-- For tables, try recursive deep equality
if typeA == "table" and typeB == "table" then
local checkedKeys = {}
for key, value in a do
checkedKeys[key] = true
if not trueEquals(value, b[key]) then
return false
end
end
for key, value in b do
if checkedKeys[key] then
continue
end
if not trueEquals(value, a[key]) then
return false
end
end
return true
-- For NaN, check if both values are not equal to themselves
elseif a ~= a and b ~= b then
return true
-- For numbers, compare with epsilon of 0.0001 to avoid floating point inequality
elseif typeA == "number" and typeB == "number" then
return fuzzyEq(a, b, 0.0001)
-- For EnumItem->number, compare the EnumItem's value
elseif typeA == "number" and typeB == "EnumItem" then
return a == b.Value
elseif typeA == "EnumItem" and typeB == "number" then
return a.Value == b
-- For Color3s, compare to RGB ints to avoid floating point inequality
elseif typeA == "Color3" and typeB == "Color3" then
local aR, aG, aB = math.floor(a.R * 255), math.floor(a.G * 255), math.floor(a.B * 255)
local bR, bG, bB = math.floor(b.R * 255), math.floor(b.G * 255), math.floor(b.B * 255)
return aR == bR and aG == bG and aB == bB
-- For CFrames, compare to components with epsilon of 0.0001 to avoid floating point inequality
elseif typeA == "CFrame" and typeB == "CFrame" then
local aComponents, bComponents = { a:GetComponents() }, { b:GetComponents() }
for i, aComponent in aComponents do
if not fuzzyEq(aComponent, bComponents[i], 0.0001) then
return false
end
end
return true
-- For Vector3s, compare to components with epsilon of 0.0001 to avoid floating point inequality
elseif typeA == "Vector3" and typeB == "Vector3" then
local aComponents, bComponents = { a.X, a.Y, a.Z }, { b.X, b.Y, b.Z }
for i, aComponent in aComponents do
if not fuzzyEq(aComponent, bComponents[i], 0.0001) then
return false
end
end
return true
-- For Vector2s, compare to components with epsilon of 0.0001 to avoid floating point inequality
elseif typeA == "Vector2" and typeB == "Vector2" then
local aComponents, bComponents = { a.X, a.Y }, { b.X, b.Y }
for i, aComponent in aComponents do
if not fuzzyEq(aComponent, bComponents[i], 0.0001) then
return false
end
end
return true
end
return false
end
return trueEquals

View File

@@ -18,6 +18,7 @@ local PatchSet = require(script.Parent.PatchSet)
local Reconciler = require(script.Parent.Reconciler) local Reconciler = require(script.Parent.Reconciler)
local strict = require(script.Parent.strict) local strict = require(script.Parent.strict)
local Settings = require(script.Parent.Settings) local Settings = require(script.Parent.Settings)
local orderSwaps = require(script.Parent.orderSwaps)
local Status = strict("Session.Status", { local Status = strict("Session.Status", {
NotStarted = "NotStarted", NotStarted = "NotStarted",
@@ -320,6 +321,14 @@ function ServeSession:__replaceInstances(idList)
return false return false
end end
-- Roblox appends to GetChildren() on every reparent, so the order in which
-- we re-parent replacements determines their final sibling order.
-- We process ancestors before descendants (so each replacement's
-- parent already exists when we re-parent it) and siblings in their original
-- GetChildren() order. Because the loop below moves the old instance's
-- children into the replacement *before* re-parenting the replacement, this
-- rebuilds GetChildren() exactly as it was before the swap.
local swaps = {}
for id, replacement in replacements do for id, replacement in replacements do
local oldInstance = self.__instanceMap.fromIds[id] local oldInstance = self.__instanceMap.fromIds[id]
if not oldInstance then if not oldInstance then
@@ -328,6 +337,16 @@ function ServeSession:__replaceInstances(idList)
continue continue
end end
table.insert(swaps, {
id = id,
replacement = replacement,
oldInstance = oldInstance,
})
end
for _, swap in orderSwaps(swaps) do
local id, replacement, oldInstance = swap.id, swap.replacement, swap.oldInstance
self.__instanceMap:insert(id, replacement) self.__instanceMap:insert(id, replacement)
Log.trace("Swapping Instance {} out via api/models/ endpoint", id) Log.trace("Swapping Instance {} out via api/models/ endpoint", id)
local oldParent = oldInstance.Parent local oldParent = oldInstance.Parent

44
plugin/src/orderSwaps.lua Normal file
View File

@@ -0,0 +1,44 @@
--[[
Determines the order in which `ServeSession:__replaceInstances` should swap
instances so that sibling order is preserved.
Roblox appends to `GetChildren()` on every reparent, so the order in which we
re-parent replacements determines their final sibling order. To rebuild
`GetChildren()` exactly as it was before the swap we must:
* process ancestors before descendants, so each replacement's parent already
exists when we re-parent the replacement, and
* process siblings in their original `GetChildren()` order.
`swaps` is an array of `{ id, replacement, oldInstance }` entries. This sorts
the array in place (annotating each entry with `depth`/`siblingIndex`) and
returns it.
]]
local function orderSwaps(swaps)
for _, swap in swaps do
local depth = 0
local ancestor = swap.oldInstance.Parent
while ancestor ~= nil do
depth += 1
ancestor = ancestor.Parent
end
swap.depth = depth
local siblingIndex = 0
if swap.oldInstance.Parent ~= nil then
siblingIndex = table.find(swap.oldInstance.Parent:GetChildren(), swap.oldInstance) or 0
end
swap.siblingIndex = siblingIndex
end
table.sort(swaps, function(a, b)
if a.depth ~= b.depth then
return a.depth < b.depth
end
return a.siblingIndex < b.siblingIndex
end)
return swaps
end
return orderSwaps

View File

@@ -0,0 +1,57 @@
return function()
local orderSwaps = require(script.Parent.orderSwaps)
it("orders same-named siblings by their original GetChildren order", function()
local parent = Instance.new("Model")
local a1 = Instance.new("Part")
a1.Name = "a"
a1.Parent = parent
local a2 = Instance.new("Part")
a2.Name = "a"
a2.Parent = parent
local a3 = Instance.new("Part")
a3.Name = "a"
a3.Parent = parent
-- Input deliberately out of sibling order.
-- orderSwaps must restore the GetChildren() order.
local ordered = orderSwaps({
{ id = "3", oldInstance = a3 },
{ id = "1", oldInstance = a1 },
{ id = "2", oldInstance = a2 },
})
expect(ordered[1].oldInstance).to.equal(a1)
expect(ordered[2].oldInstance).to.equal(a2)
expect(ordered[3].oldInstance).to.equal(a3)
end)
it("orders ancestors before descendants", function()
local root = Instance.new("Model")
local child = Instance.new("Folder")
child.Parent = root
local grandchild = Instance.new("Part")
grandchild.Parent = child
local ordered = orderSwaps({
{ id = "grandchild", oldInstance = grandchild },
{ id = "child", oldInstance = child },
{ id = "root", oldInstance = root },
})
expect(ordered[1].oldInstance).to.equal(root)
expect(ordered[2].oldInstance).to.equal(child)
expect(ordered[3].oldInstance).to.equal(grandchild)
end)
it("returns a single swap unchanged", function()
local part = Instance.new("Part")
local ordered = orderSwaps({
{ id = "1", oldInstance = part },
})
expect(#ordered).to.equal(1)
expect(ordered[1].oldInstance).to.equal(part)
end)
end

View File

@@ -0,0 +1,27 @@
---
source: tests/tests/build.rs
expression: contents
---
<roblox version="4">
<Item class="Folder" referent="0">
<Properties>
<string name="Name">plugin_init</string>
</Properties>
<Item class="Script" referent="1">
<Properties>
<string name="Name">lua</string>
<token name="RunContext">3</token>
<string name="Source"><![CDATA[return "From folder/lua/init.plugin.lua"
]]></string>
</Properties>
</Item>
<Item class="Script" referent="2">
<Properties>
<string name="Name">luau</string>
<token name="RunContext">3</token>
<string name="Source"><![CDATA[return "From folder/luau/init.plugin.luau"
]]></string>
</Properties>
</Item>
</Item>
</roblox>

View File

@@ -0,0 +1,6 @@
{
"name": "plugin_init",
"tree": {
"$path": "folder"
}
}

View File

@@ -0,0 +1 @@
return "From folder/lua/init.plugin.lua"

View File

@@ -0,0 +1 @@
return "From folder/luau/init.plugin.luau"

View File

@@ -0,0 +1,5 @@
---
source: tests/rojo_test/syncback_util.rs
expression: "String::from_utf8_lossy(&output.stdout)"
---
Writing default.project.json

View File

@@ -0,0 +1,59 @@
---
source: tests/tests/syncback.rs
expression: default.project.json
---
{
"name": "SyncbackTest",
"tree": {
"$className": "DataModel",
"Workspace": {
"$className": "Workspace",
"TestPart": {
"$className": "Part",
"$properties": {
"Anchored": true,
"Color": {
"Color3uint8": [
0,
0,
255
]
}
}
},
"$properties": {
"EnableSLIMAvatars": {
"Enum": 0
},
"ImprovedAnimationConstraint": {
"Enum": 0
},
"ImprovedPhysicsReplication": {
"Enum": 0
},
"LayeredClothingCacheOptimizations": {
"Enum": 0
},
"MeshStreamingAndImprovedLods": {
"Enum": 0
},
"NextGenerationReplication": {
"Enum": 0
},
"PlayerScriptsUseInputActionSystem": {
"Enum": 0
},
"UseFixedSimulation": {
"Enum": 0
},
"UseNewLuauTypeSolver": "Disabled",
"ValidateEnabledProximityPrompt": {
"Enum": 0
}
},
"$attributes": {
"Rojo_Target_CurrentCamera": "302d573157260ee80a3baa32000003b5"
}
}
}
}

View File

@@ -0,0 +1,21 @@
{
"name": "SyncbackTest",
"tree": {
"$className": "DataModel",
"Workspace": {
"$className": "Workspace",
"TestPart": {
"$className": "Part",
"$properties": {
"Transparency": 1.0,
"Anchored": true,
"Color": [
1.0,
0.0,
0.0
]
}
}
}
}
}

View File

@@ -3,3 +3,4 @@ rojo = "rojo-rbx/rojo@7.5.1"
selene = "Kampfkarren/selene@0.29.0" selene = "Kampfkarren/selene@0.29.0"
stylua = "JohnnyMorganz/stylua@2.1.0" stylua = "JohnnyMorganz/stylua@2.1.0"
run-in-roblox = "rojo-rbx/run-in-roblox@0.3.0" run-in-roblox = "rojo-rbx/run-in-roblox@0.3.0"
lune = "lune-org/lune@0.10.4"

View File

@@ -1,12 +1,12 @@
use std::{
fs,
sync::{Arc, Mutex},
};
use crossbeam_channel::{select, Receiver, RecvError, Sender}; use crossbeam_channel::{select, Receiver, RecvError, Sender};
use jod_thread::JoinHandle; use jod_thread::JoinHandle;
use memofs::{IoResultExt, Vfs, VfsEvent}; use memofs::{IoResultExt, Vfs, VfsEvent};
use rbx_dom_weak::types::{Ref, Variant}; use rbx_dom_weak::types::{Ref, Variant};
use std::path::PathBuf;
use std::{
fs,
sync::{Arc, Mutex},
};
use crate::{ use crate::{
message_queue::MessageQueue, message_queue::MessageQueue,
@@ -114,6 +114,49 @@ struct JobThreadContext {
} }
impl JobThreadContext { impl JobThreadContext {
/// Computes and applies patches to the DOM for a given file path.
///
/// This function finds the nearest ancestor to the given path that has associated instances
/// in the tree.
/// It then computes and applies changes for each affected instance ID and
/// returns a vector of applied patch sets.
fn apply_patches(&self, path: PathBuf) -> Vec<AppliedPatchSet> {
let mut tree = self.tree.lock().unwrap();
let mut applied_patches = Vec::new();
// Find the nearest ancestor to this path that has
// associated instances in the tree. This helps make sure
// that we handle additions correctly, especially if we
// receive events for descendants of a large tree being
// created all at once.
let mut current_path = path.as_path();
let affected_ids = loop {
let ids = tree.get_ids_at_path(current_path);
log::trace!("Path {} affects IDs {:?}", current_path.display(), ids);
if !ids.is_empty() {
break ids.to_vec();
}
log::trace!("Trying parent path...");
match current_path.parent() {
Some(parent) => current_path = parent,
None => break Vec::new(),
}
};
for id in affected_ids {
if let Some(patch) = compute_and_apply_changes(&mut tree, &self.vfs, id) {
if !patch.is_empty() {
applied_patches.push(patch);
}
}
}
applied_patches
}
fn handle_vfs_event(&self, event: VfsEvent) { fn handle_vfs_event(&self, event: VfsEvent) {
log::trace!("Vfs event: {:?}", event); log::trace!("Vfs event: {:?}", event);
@@ -125,41 +168,16 @@ impl JobThreadContext {
// For a given VFS event, we might have many changes to different parts // For a given VFS event, we might have many changes to different parts
// of the tree. Calculate and apply all of these changes. // of the tree. Calculate and apply all of these changes.
let applied_patches = match event { let applied_patches = match event {
VfsEvent::Create(path) | VfsEvent::Remove(path) | VfsEvent::Write(path) => { VfsEvent::Create(path) | VfsEvent::Write(path) => {
let mut tree = self.tree.lock().unwrap(); self.apply_patches(self.vfs.canonicalize(&path).unwrap())
let mut applied_patches = Vec::new(); }
VfsEvent::Remove(path) => {
// Find the nearest ancestor to this path that has // MemoFS does not track parent removals yet, so we can canonicalize
// associated instances in the tree. This helps make sure // the parent path safely and then append the removed path's file name.
// that we handle additions correctly, especially if we let parent = path.parent().unwrap();
// receive events for descendants of a large tree being let file_name = path.file_name().unwrap();
// created all at once. let parent_normalized = self.vfs.canonicalize(parent).unwrap();
let mut current_path = path.as_path(); self.apply_patches(parent_normalized.join(file_name))
let affected_ids = loop {
let ids = tree.get_ids_at_path(current_path);
log::trace!("Path {} affects IDs {:?}", current_path.display(), ids);
if !ids.is_empty() {
break ids.to_vec();
}
log::trace!("Trying parent path...");
match current_path.parent() {
Some(parent) => current_path = parent,
None => break Vec::new(),
}
};
for id in affected_ids {
if let Some(patch) = compute_and_apply_changes(&mut tree, &self.vfs, id) {
if !patch.is_empty() {
applied_patches.push(patch);
}
}
}
applied_patches
} }
_ => { _ => {
log::warn!("Unhandled VFS event: {:?}", event); log::warn!("Unhandled VFS event: {:?}", event);
@@ -182,7 +200,15 @@ impl JobThreadContext {
if let Some(instance) = tree.get_instance(id) { if let Some(instance) = tree.get_instance(id) {
if let Some(instigating_source) = &instance.metadata().instigating_source { if let Some(instigating_source) = &instance.metadata().instigating_source {
match instigating_source { match instigating_source {
InstigatingSource::Path(path) => fs::remove_file(path).unwrap(), InstigatingSource::Path(path) => {
if let Err(err) = fs::remove_file(path) {
log::error!(
"Failed to remove file {}: {}",
path.display(),
err
);
}
}
InstigatingSource::ProjectNode { .. } => { InstigatingSource::ProjectNode { .. } => {
log::warn!( log::warn!(
"Cannot remove instance {:?}, it's from a project file", "Cannot remove instance {:?}, it's from a project file",
@@ -226,7 +252,13 @@ impl JobThreadContext {
match instigating_source { match instigating_source {
InstigatingSource::Path(path) => { InstigatingSource::Path(path) => {
if let Some(Variant::String(value)) = changed_value { if let Some(Variant::String(value)) = changed_value {
fs::write(path, value).unwrap(); if let Err(err) = fs::write(path, value) {
log::error!(
"Failed to write file {}: {}",
path.display(),
err
);
}
} else { } else {
log::warn!("Cannot change Source to non-string value."); log::warn!("Cannot change Source to non-string value.");
} }

View File

@@ -75,10 +75,10 @@ impl BuildCommand {
_ => unreachable!(), _ => unreachable!(),
}; };
let project_path = resolve_path(&self.project); let project_path = resolve_path(&self.project)?;
log::trace!("Constructing in-memory filesystem"); log::trace!("Constructing in-memory filesystem");
let vfs = Vfs::new_default(); let vfs = Vfs::new_default()?;
vfs.set_watch_enabled(self.watch); vfs.set_watch_enabled(self.watch);
let session = ServeSession::new(vfs, project_path)?; let session = ServeSession::new(vfs, project_path)?;
@@ -87,11 +87,16 @@ impl BuildCommand {
write_model(&session, &output_path, output_kind)?; write_model(&session, &output_path, output_kind)?;
if self.watch { if self.watch {
let rt = Runtime::new().unwrap(); let rt = Runtime::new().context("Failed to start the async runtime for watch mode")?;
loop { loop {
let receiver = session.message_queue().subscribe(cursor); let receiver = session.message_queue().subscribe(cursor);
let (new_cursor, _patch_set) = rt.block_on(receiver).unwrap(); let (new_cursor, _patch_set) = match rt.block_on(receiver) {
Ok(message) => message,
// The message queue was dropped, so there is nothing left
// to watch. Stop watching gracefully.
Err(_) => break,
};
cursor = new_cursor; cursor = new_cursor;
write_model(&session, &output_path, output_kind)?; write_model(&session, &output_path, output_kind)?;

View File

@@ -18,10 +18,10 @@ pub struct FmtProjectCommand {
impl FmtProjectCommand { impl FmtProjectCommand {
pub fn run(self) -> anyhow::Result<()> { pub fn run(self) -> anyhow::Result<()> {
let vfs = Vfs::new_default(); let vfs = Vfs::new_default()?;
vfs.set_watch_enabled(false); vfs.set_watch_enabled(false);
let base_path = resolve_path(&self.project); let base_path = resolve_path(&self.project)?;
let project = Project::load_fuzzy(&vfs, &base_path)? let project = Project::load_fuzzy(&vfs, &base_path)?
.context("A project file is required to run 'rojo fmt-project'")?; .context("A project file is required to run 'rojo fmt-project'")?;

View File

@@ -9,7 +9,7 @@ use std::{
io::{self, Write}, io::{self, Write},
}; };
use anyhow::{bail, format_err}; use anyhow::{bail, format_err, Context};
use clap::Parser; use clap::Parser;
use fs_err as fs; use fs_err as fs;
use fs_err::OpenOptions; use fs_err::OpenOptions;
@@ -42,9 +42,9 @@ pub struct InitCommand {
impl InitCommand { impl InitCommand {
pub fn run(self) -> anyhow::Result<()> { pub fn run(self) -> anyhow::Result<()> {
let template = self.kind.template(); let template = self.kind.template()?;
let base_path = resolve_path(&self.path); let base_path = resolve_path(&self.path)?;
fs::create_dir_all(&base_path)?; fs::create_dir_all(&base_path)?;
let canonical = fs::canonicalize(&base_path)?; let canonical = fs::canonicalize(&base_path)?;
@@ -128,7 +128,7 @@ pub enum InitKind {
} }
impl InitKind { impl InitKind {
fn template(&self) -> InMemoryFs { fn template(&self) -> anyhow::Result<InMemoryFs> {
let template_path = match self { let template_path = match self {
Self::Place => "place", Self::Place => "place",
Self::Model => "model", Self::Model => "model",
@@ -136,20 +136,24 @@ impl InitKind {
}; };
let snapshot: VfsSnapshot = bincode::deserialize(TEMPLATE_BINCODE) let snapshot: VfsSnapshot = bincode::deserialize(TEMPLATE_BINCODE)
.expect("Rojo's templates were not properly packed into Rojo's binary"); .context("Rojo's templates were not properly packed into Rojo's binary. This is a bug in Rojo; please file an issue.")?;
if let VfsSnapshot::Dir { mut children } = snapshot { let VfsSnapshot::Dir { mut children } = snapshot else {
if let Some(template) = children.remove(template_path) { bail!("Rojo's templates were packed as a file instead of a directory. This is a bug in Rojo; please file an issue.");
let mut fs = InMemoryFs::new(); };
fs.load_snapshot("", template)
.expect("loading a template in memory should never fail"); let template = children.remove(template_path).ok_or_else(|| {
fs format_err!(
} else { "The template for project type {:?} is missing. This is a bug in Rojo; please file an issue.",
panic!("template for project type {:?} is missing", self) self
} )
} else { })?;
panic!("Rojo's templates were packed as a file instead of a directory")
} let mut fs = InMemoryFs::new();
fs.load_snapshot("", template)
.context("Failed to load Rojo's bundled template into memory")?;
Ok(fs)
} }
} }

View File

@@ -12,6 +12,7 @@ mod upload;
use std::{borrow::Cow, env, path::Path, str::FromStr}; use std::{borrow::Cow, env, path::Path, str::FromStr};
use anyhow::Context;
use clap::Parser; use clap::Parser;
use thiserror::Error; use thiserror::Error;
@@ -125,10 +126,14 @@ pub enum Subcommand {
Syncback(SyncbackCommand), Syncback(SyncbackCommand),
} }
pub(super) fn resolve_path(path: &Path) -> Cow<'_, Path> { pub(super) fn resolve_path(path: &Path) -> anyhow::Result<Cow<'_, Path>> {
if path.is_absolute() { if path.is_absolute() {
Cow::Borrowed(path) Ok(Cow::Borrowed(path))
} else { } else {
Cow::Owned(env::current_dir().unwrap().join(path)) let current_dir = env::current_dir().context(
"Could not determine the current working directory. \
It may have been deleted, or Rojo may not have permission to access it.",
)?;
Ok(Cow::Owned(current_dir.join(path)))
} }
} }

View File

@@ -98,5 +98,5 @@ fn uninstall_plugin() -> anyhow::Result<()> {
#[test] #[test]
fn plugin_initialize() { fn plugin_initialize() {
assert!(initialize_plugin().is_ok()) let _ = initialize_plugin().unwrap();
} }

View File

@@ -31,13 +31,21 @@ pub struct ServeCommand {
/// it has none. /// it has none.
#[clap(long)] #[clap(long)]
pub port: Option<u16>, pub port: Option<u16>,
/// Extra `Host`/`Origin` values the server will accept, beyond localhost and
/// the bind address (for example a hostname like `mypc.lan`). Repeat the
/// option or comma-separate to allow several. When given, this overrides the
/// project's `serveAllowedHosts`. Listing any host also turns on Host/Origin
/// validation for binds where it is otherwise off (such as `0.0.0.0`).
#[clap(long, value_delimiter = ',')]
pub allowed_hosts: Vec<String>,
} }
impl ServeCommand { impl ServeCommand {
pub fn run(self, global: GlobalOptions) -> anyhow::Result<()> { pub fn run(self, global: GlobalOptions) -> anyhow::Result<()> {
let project_path = resolve_path(&self.project); let project_path = resolve_path(&self.project)?;
let vfs = Vfs::new_default(); let vfs = Vfs::new_default()?;
let session = Arc::new(ServeSession::new(vfs, project_path)?); let session = Arc::new(ServeSession::new(vfs, project_path)?);
@@ -51,10 +59,19 @@ impl ServeCommand {
.or_else(|| session.project_port()) .or_else(|| session.project_port())
.unwrap_or(DEFAULT_PORT); .unwrap_or(DEFAULT_PORT);
// The CLI flag, when given, replaces the project's list rather than
// merging with it, matching how --address and --port override theirs.
let allowed_hosts = if self.allowed_hosts.is_empty() {
session.serve_allowed_hosts().to_vec()
} else {
self.allowed_hosts
};
let server = LiveServer::new(session); let server = LiveServer::new(session);
let _ = show_start_message(ip, port, global.color.into()); server.start((ip, port).into(), allowed_hosts, || {
server.start((ip, port).into()); let _ = show_start_message(ip, port, global.color.into());
})?;
Ok(()) Ok(())
} }
@@ -86,6 +103,25 @@ fn show_start_message(bind_address: IpAddr, port: u16, color: ColorChoice) -> io
writeln!(&mut buffer)?; writeln!(&mut buffer)?;
if !bind_address.is_loopback() {
let mut warning = ColorSpec::new();
warning.set_fg(Some(Color::Yellow)).set_bold(true);
buffer.set_color(&warning)?;
writeln!(
&mut buffer,
"WARNING: This server is bound to {address_string}, which is reachable from the \
network.\n\
The serve API is unauthenticated, so anyone who can reach {address_string}:{port} \
can read\n\
and modify your project's source. Prefer binding to localhost and tunneling (e.g. \
SSH,\n\
Tailscale, or WireGuard) when you need remote access."
)?;
buffer.set_color(&ColorSpec::new())?;
writeln!(&mut buffer)?;
}
buffer.set_color(&ColorSpec::new())?; buffer.set_color(&ColorSpec::new())?;
write!(&mut buffer, "Visit ")?; write!(&mut buffer, "Visit ")?;

View File

@@ -0,0 +1,35 @@
---
source: src/cli/sourcemap.rs
expression: sourcemap_contents
---
{
"name": "default",
"className": "DataModel",
"filePaths": "[...1 path omitted...]",
"children": [
{
"name": "ReplicatedStorage",
"className": "ReplicatedStorage",
"children": [
{
"name": "Project",
"className": "ModuleScript",
"filePaths": "[...1 path omitted...]",
"children": [
{
"name": "Module",
"className": "Folder",
"children": [
{
"name": "module",
"className": "ModuleScript",
"filePaths": "[...1 path omitted...]"
}
]
}
]
}
]
}
]
}

View File

@@ -0,0 +1,41 @@
---
source: src/cli/sourcemap.rs
expression: sourcemap_contents
---
{
"name": "default",
"className": "DataModel",
"filePaths": [
"default.project.json"
],
"children": [
{
"name": "ReplicatedStorage",
"className": "ReplicatedStorage",
"children": [
{
"name": "Project",
"className": "ModuleScript",
"filePaths": [
"src/init.luau"
],
"children": [
{
"name": "Module",
"className": "Folder",
"children": [
{
"name": "module",
"className": "ModuleScript",
"filePaths": [
"../module/module.luau"
]
}
]
}
]
}
]
}
]
}

View File

@@ -5,12 +5,13 @@ use std::{
path::{self, Path, PathBuf}, path::{self, Path, PathBuf},
}; };
use anyhow::Context;
use clap::Parser; use clap::Parser;
use fs_err::File; use fs_err::File;
use memofs::Vfs; use memofs::Vfs;
use rayon::prelude::*; use rayon::prelude::*;
use rbx_dom_weak::{types::Ref, Ustr}; use rbx_dom_weak::{types::Ref, Ustr};
use serde::Serialize; use serde::{Deserialize, Serialize};
use tokio::runtime::Runtime; use tokio::runtime::Runtime;
use crate::{ use crate::{
@@ -24,19 +25,20 @@ const PATH_STRIP_FAILED_ERR: &str = "Failed to create relative paths for project
const ABSOLUTE_PATH_FAILED_ERR: &str = "Failed to turn relative path into absolute path!"; const ABSOLUTE_PATH_FAILED_ERR: &str = "Failed to turn relative path into absolute path!";
/// Representation of a node in the generated sourcemap tree. /// Representation of a node in the generated sourcemap tree.
#[derive(Serialize)] #[derive(Serialize, Deserialize)]
#[serde(rename_all = "camelCase")] #[serde(rename_all = "camelCase")]
struct SourcemapNode<'a> { struct SourcemapNode<'a> {
name: &'a str, name: &'a str,
class_name: Ustr, class_name: Ustr,
#[serde( #[serde(
default,
skip_serializing_if = "Vec::is_empty", skip_serializing_if = "Vec::is_empty",
serialize_with = "crate::path_serializer::serialize_vec_absolute" serialize_with = "crate::path_serializer::serialize_vec_absolute"
)] )]
file_paths: Vec<Cow<'a, Path>>, file_paths: Vec<Cow<'a, Path>>,
#[serde(skip_serializing_if = "Vec::is_empty")] #[serde(default, skip_serializing_if = "Vec::is_empty")]
children: Vec<SourcemapNode<'a>>, children: Vec<SourcemapNode<'a>>,
} }
@@ -70,12 +72,13 @@ pub struct SourcemapCommand {
impl SourcemapCommand { impl SourcemapCommand {
pub fn run(self) -> anyhow::Result<()> { pub fn run(self) -> anyhow::Result<()> {
let project_path = resolve_path(&self.project); let project_path = fs_err::canonicalize(resolve_path(&self.project)?)?;
log::trace!("Constructing in-memory filesystem"); log::trace!("Constructing filesystem with StdBackend");
let vfs = Vfs::new_default(); let vfs = Vfs::new_default()?;
vfs.set_watch_enabled(self.watch); vfs.set_watch_enabled(self.watch);
log::trace!("Setting up session for sourcemap generation");
let session = ServeSession::new(vfs, project_path)?; let session = ServeSession::new(vfs, project_path)?;
let mut cursor = session.message_queue().cursor(); let mut cursor = session.message_queue().cursor();
@@ -87,19 +90,27 @@ impl SourcemapCommand {
// Pre-build a rayon threadpool with a low number of threads to avoid // Pre-build a rayon threadpool with a low number of threads to avoid
// dynamic creation overhead on systems with a high number of cpus. // dynamic creation overhead on systems with a high number of cpus.
log::trace!("Setting rayon global threadpool");
rayon::ThreadPoolBuilder::new() rayon::ThreadPoolBuilder::new()
.num_threads(num_cpus::get().min(6)) .num_threads(num_cpus::get().min(6))
.build_global() .build_global()
.unwrap(); .ok();
log::trace!("Writing initial sourcemap");
write_sourcemap(&session, self.output.as_deref(), filter, self.absolute)?; write_sourcemap(&session, self.output.as_deref(), filter, self.absolute)?;
if self.watch { if self.watch {
let rt = Runtime::new().unwrap(); log::trace!("Setting up runtime for watch mode");
let rt = Runtime::new().context("Failed to start the async runtime for watch mode")?;
loop { loop {
let receiver = session.message_queue().subscribe(cursor); let receiver = session.message_queue().subscribe(cursor);
let (new_cursor, patch_set) = rt.block_on(receiver).unwrap(); let (new_cursor, patch_set) = match rt.block_on(receiver) {
Ok(message) => message,
// The message queue was dropped, so there is nothing left
// to watch. Stop watching gracefully.
Err(_) => break,
};
cursor = new_cursor; cursor = new_cursor;
if patch_set_affects_sourcemap(&session, &patch_set, filter) { if patch_set_affects_sourcemap(&session, &patch_set, filter) {
@@ -208,7 +219,7 @@ fn recurse_create_node<'a>(
} else { } else {
for val in file_paths { for val in file_paths {
output_file_paths.push(Cow::from( output_file_paths.push(Cow::from(
val.strip_prefix(project_dir).expect(PATH_STRIP_FAILED_ERR), pathdiff::diff_paths(val, project_dir).expect(PATH_STRIP_FAILED_ERR),
)); ));
} }
}; };
@@ -250,3 +261,80 @@ fn write_sourcemap(
Ok(()) Ok(())
} }
#[cfg(test)]
mod test {
use crate::cli::sourcemap::SourcemapNode;
use crate::cli::SourcemapCommand;
use insta::internals::Content;
use std::path::Path;
#[test]
fn maps_relative_paths() {
let sourcemap_dir = tempfile::tempdir().unwrap();
let sourcemap_output = sourcemap_dir.path().join("sourcemap.json");
let project_path = fs_err::canonicalize(
Path::new(env!("CARGO_MANIFEST_DIR"))
.join("test-projects")
.join("relative_paths")
.join("project"),
)
.unwrap();
let sourcemap_command = SourcemapCommand {
project: project_path,
output: Some(sourcemap_output.clone()),
include_non_scripts: false,
watch: false,
absolute: false,
};
assert!(sourcemap_command.run().is_ok());
let raw_sourcemap_contents = fs_err::read_to_string(sourcemap_output.as_path()).unwrap();
let sourcemap_contents =
serde_json::from_str::<SourcemapNode>(&raw_sourcemap_contents).unwrap();
insta::assert_json_snapshot!(sourcemap_contents);
}
#[test]
fn maps_absolute_paths() {
let sourcemap_dir = tempfile::tempdir().unwrap();
let sourcemap_output = sourcemap_dir.path().join("sourcemap.json");
let project_path = fs_err::canonicalize(
Path::new(env!("CARGO_MANIFEST_DIR"))
.join("test-projects")
.join("relative_paths")
.join("project"),
)
.unwrap();
let sourcemap_command = SourcemapCommand {
project: project_path,
output: Some(sourcemap_output.clone()),
include_non_scripts: false,
watch: false,
absolute: true,
};
assert!(sourcemap_command.run().is_ok());
let raw_sourcemap_contents = fs_err::read_to_string(sourcemap_output.as_path()).unwrap();
let sourcemap_contents =
serde_json::from_str::<SourcemapNode>(&raw_sourcemap_contents).unwrap();
insta::assert_json_snapshot!(sourcemap_contents, {
".**.filePaths" => insta::dynamic_redaction(|mut value, _path| {
let mut paths_count = 0;
match value {
Content::Seq(ref mut vec) => {
for path in vec.iter().map(|i| i.as_str().unwrap()) {
assert_eq!(fs_err::canonicalize(path).is_ok(), true, "path was not valid");
assert_eq!(Path::new(path).is_absolute(), true, "path was not absolute");
paths_count += 1;
}
}
_ => panic!("Expected filePaths to be a sequence"),
}
format!("[...{} path{} omitted...]", paths_count, if paths_count != 1 { "s" } else { "" } )
})
});
}
}

View File

@@ -58,8 +58,8 @@ pub struct SyncbackCommand {
impl SyncbackCommand { impl SyncbackCommand {
pub fn run(&self, global: GlobalOptions) -> anyhow::Result<()> { pub fn run(&self, global: GlobalOptions) -> anyhow::Result<()> {
let path_old = resolve_path(&self.project); let path_old = resolve_path(&self.project)?;
let path_new = resolve_path(&self.input); let path_new = resolve_path(&self.input)?;
let input_kind = FileKind::from_path(&path_new).context(UNKNOWN_INPUT_KIND_ERR)?; let input_kind = FileKind::from_path(&path_new).context(UNKNOWN_INPUT_KIND_ERR)?;
let dom_start_timer = Instant::now(); let dom_start_timer = Instant::now();
@@ -69,7 +69,7 @@ impl SyncbackCommand {
dom_start_timer.elapsed().as_secs_f32() dom_start_timer.elapsed().as_secs_f32()
); );
let vfs = Vfs::new_default(); let vfs = Vfs::new_default()?;
vfs.set_watch_enabled(false); vfs.set_watch_enabled(false);
let project_start_timer = Instant::now(); let project_start_timer = Instant::now();

View File

@@ -38,9 +38,9 @@ pub struct UploadCommand {
impl UploadCommand { impl UploadCommand {
pub fn run(self) -> Result<(), anyhow::Error> { pub fn run(self) -> Result<(), anyhow::Error> {
let project_path = resolve_path(&self.project); let project_path = resolve_path(&self.project)?;
let vfs = Vfs::new_default(); let vfs = Vfs::new_default()?;
let session = ServeSession::new(vfs, project_path)?; let session = ServeSession::new(vfs, project_path)?;

View File

@@ -48,3 +48,61 @@ impl<'de> Deserialize<'de> for Glob {
Glob::new(&glob).map_err(D::Error::custom) Glob::new(&glob).map_err(D::Error::custom)
} }
} }
/// A glob with optional gitignore-style negation. A leading `!` marks the
/// pattern as a negation (re-includes paths that an earlier rule excluded).
/// To match a literal `!` at the start of a pattern, escape it with `\!`.
#[derive(Debug, Clone)]
pub struct IgnorableGlob {
glob: Glob,
negated: bool,
raw: String,
}
impl IgnorableGlob {
pub fn new(pattern: &str) -> Result<Self, Error> {
let (negated, body) = if let Some(rest) = pattern.strip_prefix('!') {
(true, rest)
} else if pattern.starts_with(r"\!") {
(false, &pattern[1..])
} else {
(false, pattern)
};
Ok(IgnorableGlob {
glob: Glob::new(body)?,
negated,
raw: pattern.to_owned(),
})
}
pub fn is_match<P: AsRef<Path>>(&self, path: P) -> bool {
self.glob.is_match(path)
}
pub fn is_negation(&self) -> bool {
self.negated
}
}
impl PartialEq for IgnorableGlob {
fn eq(&self, other: &Self) -> bool {
self.negated == other.negated && self.glob == other.glob
}
}
impl Eq for IgnorableGlob {}
impl Serialize for IgnorableGlob {
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
serializer.serialize_str(&self.raw)
}
}
impl<'de> Deserialize<'de> for IgnorableGlob {
fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
let pattern = String::deserialize(deserializer)?;
IgnorableGlob::new(&pattern).map_err(D::Error::custom)
}
}

View File

@@ -12,7 +12,8 @@ use serde::{Deserialize, Serialize};
use thiserror::Error; use thiserror::Error;
use crate::{ use crate::{
glob::Glob, json, resolution::UnresolvedValue, snapshot::SyncRule, syncback::SyncbackRules, glob::IgnorableGlob, json, resolution::UnresolvedValue, snapshot::SyncRule,
syncback::SyncbackRules,
}; };
/// Represents 'default' project names that act as `init` files /// Represents 'default' project names that act as `init` files
@@ -105,6 +106,15 @@ pub struct Project {
#[serde(skip_serializing_if = "Option::is_none")] #[serde(skip_serializing_if = "Option::is_none")]
pub serve_address: Option<IpAddr>, pub serve_address: Option<IpAddr>,
/// Additional `Host`/`Origin` header values that `rojo serve` will accept
/// beyond `localhost` and the bind address, such as a hostname like
/// `mypc.lan` used to reach a network-exposed server by name. Listing any
/// host also turns on `Host`/`Origin` validation for binds where it is
/// otherwise off (such as `0.0.0.0`). The `--allowed-hosts` CLI option
/// overrides this field when provided.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub serve_allowed_hosts: Vec<String>,
/// Determines if Rojo should emit scripts with the appropriate `RunContext` /// Determines if Rojo should emit scripts with the appropriate `RunContext`
/// for `*.client.lua` and `*.server.lua` files in the project instead of /// for `*.client.lua` and `*.server.lua` files in the project instead of
/// using `Script` and `LocalScript` Instances. /// using `Script` and `LocalScript` Instances.
@@ -114,7 +124,7 @@ pub struct Project {
/// A list of globs, relative to the folder the project file is in, that /// A list of globs, relative to the folder the project file is in, that
/// match files that should be excluded if Rojo encounters them. /// match files that should be excluded if Rojo encounters them.
#[serde(default, skip_serializing_if = "Vec::is_empty")] #[serde(default, skip_serializing_if = "Vec::is_empty")]
pub glob_ignore_paths: Vec<Glob>, pub glob_ignore_paths: Vec<IgnorableGlob>,
/// A list of rules for syncback with this project file. /// A list of rules for syncback with this project file.
#[serde(skip_serializing_if = "Option::is_none")] #[serde(skip_serializing_if = "Option::is_none")]
@@ -593,4 +603,90 @@ mod test {
assert!(project.sync_rules[0].include.is_match("data.data.json")); assert!(project.sync_rules[0].include.is_match("data.data.json"));
assert!(project.sync_rules[1].include.is_match("init.module.lua")); assert!(project.sync_rules[1].include.is_match("init.module.lua"));
} }
#[test]
fn project_with_serve_allowed_hosts() {
let project_json = r#"{
"name": "TestProject",
"tree": { "$path": "src" },
"serveAllowedHosts": ["mypc.lan", "192.168.1.5"]
}"#;
let project = Project::load_from_slice(
project_json.as_bytes(),
PathBuf::from("/test/default.project.json"),
None,
)
.expect("Failed to parse project with serveAllowedHosts");
assert_eq!(project.serve_allowed_hosts, vec!["mypc.lan", "192.168.1.5"]);
}
#[test]
fn project_without_serve_allowed_hosts_defaults_to_empty() {
let project_json = r#"{
"name": "TestProject",
"tree": { "$path": "src" }
}"#;
let project = Project::load_from_slice(
project_json.as_bytes(),
PathBuf::from("/test/default.project.json"),
None,
)
.expect("Failed to parse project");
assert!(project.serve_allowed_hosts.is_empty());
}
#[test]
fn glob_ignore_paths_negation() {
let project_json = r#"{
"name": "TestProject",
"tree": { "$path": "src" },
"globIgnorePaths": [
"**/*.spec.lua",
"!keep.spec.lua",
"\\!literal.lua"
]
}"#;
let project = Project::load_from_slice(
project_json.as_bytes(),
PathBuf::from("/test/default.project.json"),
None,
)
.expect("project should parse");
let paths = &project.glob_ignore_paths;
assert_eq!(paths.len(), 3);
assert!(!paths[0].is_negation());
assert!(paths[0].is_match("foo.spec.lua"));
assert!(paths[1].is_negation());
assert!(paths[1].is_match("keep.spec.lua"));
// `\!literal.lua` should match a file literally named `!literal.lua`,
// not be parsed as a negation.
assert!(!paths[2].is_negation());
assert!(paths[2].is_match("!literal.lua"));
let rules: Vec<_> = paths
.iter()
.map(|g| crate::snapshot::PathIgnoreRule {
base_path: PathBuf::from("/test"),
glob: g.clone(),
})
.collect();
assert!(crate::snapshot::is_path_ignored(
&rules,
"/test/foo.spec.lua"
));
assert!(!crate::snapshot::is_path_ignored(
&rules,
"/test/keep.spec.lua"
));
assert!(!crate::snapshot::is_path_ignored(&rules, "/test/plain.lua"));
}
} }

View File

@@ -207,6 +207,10 @@ impl ServeSession {
self.root_project.serve_address self.root_project.serve_address
} }
pub fn serve_allowed_hosts(&self) -> &[String] {
&self.root_project.serve_allowed_hosts
}
pub fn root_dir(&self) -> &Path { pub fn root_dir(&self) -> &Path {
self.root_project.folder_location() self.root_project.folder_location()
} }

View File

@@ -8,7 +8,7 @@ use anyhow::Context;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use crate::{ use crate::{
glob::Glob, glob::{Glob, IgnorableGlob},
path_serializer, path_serializer,
project::ProjectNode, project::ProjectNode,
snapshot_middleware::{emit_legacy_scripts_default, Middleware}, snapshot_middleware::{emit_legacy_scripts_default, Middleware},
@@ -222,18 +222,37 @@ pub struct PathIgnoreRule {
pub base_path: PathBuf, pub base_path: PathBuf,
/// The actual glob that can be matched against the input path. /// The actual glob that can be matched against the input path.
pub glob: Glob, pub glob: IgnorableGlob,
} }
impl PathIgnoreRule { impl PathIgnoreRule {
pub fn passes<P: AsRef<Path>>(&self, path: P) -> bool { pub fn matches<P: AsRef<Path>>(&self, path: P) -> bool {
let path = path.as_ref(); let path = path.as_ref();
match path.strip_prefix(&self.base_path) { match path.strip_prefix(&self.base_path) {
Ok(suffix) => !self.glob.is_match(suffix), Ok(suffix) => self.glob.is_match(suffix),
Err(_) => true, Err(_) => false,
} }
} }
pub fn is_negation(&self) -> bool {
self.glob.is_negation()
}
}
/// Evaluates an ordered list of [`PathIgnoreRule`]s against a path using
/// gitignore-style "last match wins" semantics: a path is ignored if the last
/// rule whose pattern matches it is non-negated. Paths matched by no rule are
/// not ignored.
pub fn is_path_ignored<P: AsRef<Path>>(rules: &[PathIgnoreRule], path: P) -> bool {
let path = path.as_ref();
let mut ignored = false;
for rule in rules {
if rule.matches(path) {
ignored = !rule.is_negation();
}
}
ignored
} }
/// Represents where a particular Instance or InstanceSnapshot came from. /// Represents where a particular Instance or InstanceSnapshot came from.

View File

@@ -8,7 +8,7 @@ use rbx_dom_weak::{
ustr, HashMapExt as _, UstrMap, UstrSet, ustr, HashMapExt as _, UstrMap, UstrSet,
}; };
use crate::{RojoRef, REF_POINTER_ATTRIBUTE_PREFIX}; use crate::{variant_eq::variant_eq, RojoRef, REF_POINTER_ATTRIBUTE_PREFIX};
use super::{ use super::{
patch::{PatchAdd, PatchSet, PatchUpdate}, patch::{PatchAdd, PatchSet, PatchUpdate},
@@ -127,7 +127,7 @@ fn compute_property_patches(
match instance.properties().get(&name) { match instance.properties().get(&name) {
Some(instance_value) => { Some(instance_value) => {
if &snapshot_value != instance_value { if !variant_eq(&snapshot_value, instance_value) {
changed_properties.insert(name, Some(snapshot_value)); changed_properties.insert(name, Some(snapshot_value));
} }
} }

View File

@@ -42,7 +42,7 @@ pub fn snapshot_csv(
.metadata( .metadata(
InstanceMetadata::new() InstanceMetadata::new()
.instigating_source(path) .instigating_source(path)
.relevant_paths(vec![path.to_path_buf()]), .relevant_paths(vec![vfs.canonicalize(path)?]),
); );
AdjacentMetadata::read_and_apply_all(vfs, path, name, &mut snapshot)?; AdjacentMetadata::read_and_apply_all(vfs, path, name, &mut snapshot)?;
@@ -195,7 +195,7 @@ struct LocalizationEntry<'a> {
/// https://github.com/BurntSushi/rust-csv/issues/151 /// https://github.com/BurntSushi/rust-csv/issues/151
/// ///
/// This function operates in one step in order to minimize data-copying. /// This function operates in one step in order to minimize data-copying.
fn convert_localization_csv(contents: &[u8]) -> Result<String, csv::Error> { fn convert_localization_csv(contents: &[u8]) -> anyhow::Result<String> {
let mut reader = csv::Reader::from_reader(contents); let mut reader = csv::Reader::from_reader(contents);
let headers = reader.headers()?.clone(); let headers = reader.headers()?.clone();
@@ -237,7 +237,7 @@ fn convert_localization_csv(contents: &[u8]) -> Result<String, csv::Error> {
} }
let encoded = let encoded =
serde_json::to_string(&entries).expect("Could not encode JSON for localization table"); serde_json::to_string(&entries).context("Could not encode JSON for localization table")?;
Ok(encoded) Ok(encoded)
} }

View File

@@ -7,7 +7,9 @@ use anyhow::Context;
use memofs::{DirEntry, Vfs}; use memofs::{DirEntry, Vfs};
use crate::{ use crate::{
snapshot::{InstanceContext, InstanceMetadata, InstanceSnapshot, InstigatingSource}, snapshot::{
is_path_ignored, InstanceContext, InstanceMetadata, InstanceSnapshot, InstigatingSource,
},
syncback::{hash_instance, FsSnapshot, SyncbackReturn, SyncbackSnapshot}, syncback::{hash_instance, FsSnapshot, SyncbackReturn, SyncbackSnapshot},
}; };
@@ -41,12 +43,8 @@ pub fn snapshot_dir_no_meta(
path: &Path, path: &Path,
name: &str, name: &str,
) -> anyhow::Result<Option<InstanceSnapshot>> { ) -> anyhow::Result<Option<InstanceSnapshot>> {
let passes_filter_rules = |child: &DirEntry| { let passes_filter_rules =
context |child: &DirEntry| !is_path_ignored(&context.path_ignore_rules, child.path());
.path_ignore_rules
.iter()
.all(|rule| rule.passes(child.path()))
};
let mut snapshot_children = Vec::new(); let mut snapshot_children = Vec::new();
@@ -62,18 +60,21 @@ pub fn snapshot_dir_no_meta(
} }
} }
let normalized_path = vfs.canonicalize(path)?;
let relevant_paths = vec![ let relevant_paths = vec![
path.to_path_buf(), normalized_path.clone(),
// TODO: We shouldn't need to know about Lua existing in this // TODO: We shouldn't need to know about Lua existing in this
// middleware. Should we figure out a way for that function to add // middleware. Should we figure out a way for that function to add
// relevant paths to this middleware? // relevant paths to this middleware?
path.join("init.lua"), normalized_path.join("init.lua"),
path.join("init.luau"), normalized_path.join("init.luau"),
path.join("init.server.lua"), normalized_path.join("init.server.lua"),
path.join("init.server.luau"), normalized_path.join("init.server.luau"),
path.join("init.client.lua"), normalized_path.join("init.client.lua"),
path.join("init.client.luau"), normalized_path.join("init.client.luau"),
path.join("init.csv"), normalized_path.join("init.plugin.lua"),
normalized_path.join("init.plugin.luau"),
normalized_path.join("init.csv"),
]; ];
let snapshot = InstanceSnapshot::new() let snapshot = InstanceSnapshot::new()

View File

@@ -32,7 +32,7 @@ pub fn snapshot_json(
.metadata( .metadata(
InstanceMetadata::new() InstanceMetadata::new()
.instigating_source(path) .instigating_source(path)
.relevant_paths(vec![path.to_path_buf()]) .relevant_paths(vec![vfs.canonicalize(path)?])
.context(context), .context(context),
); );

View File

@@ -59,7 +59,7 @@ pub fn snapshot_json_model(
snapshot.metadata = snapshot snapshot.metadata = snapshot
.metadata .metadata
.instigating_source(path) .instigating_source(path)
.relevant_paths(vec![path.to_path_buf()]) .relevant_paths(vec![vfs.canonicalize(path)?])
.context(context) .context(context)
.specified_id(id) .specified_id(id)
.schema(schema); .schema(schema);

View File

@@ -88,7 +88,7 @@ pub fn snapshot_lua(
.metadata( .metadata(
InstanceMetadata::new() InstanceMetadata::new()
.instigating_source(path) .instigating_source(path)
.relevant_paths(vec![path.to_path_buf()]) .relevant_paths(vec![vfs.canonicalize(path)?])
.context(context), .context(context),
); );
@@ -182,6 +182,7 @@ pub fn syncback_lua_init<'sync>(
ScriptType::Server => "init.server.luau", ScriptType::Server => "init.server.luau",
ScriptType::Client => "init.client.luau", ScriptType::Client => "init.client.luau",
ScriptType::Module => "init.luau", ScriptType::Module => "init.luau",
ScriptType::Plugin => "init.plugin.luau",
_ => anyhow::bail!("syncback is not yet implemented for {script_type:?}"), _ => anyhow::bail!("syncback is not yet implemented for {script_type:?}"),
}); });

View File

@@ -91,7 +91,9 @@ pub fn snapshot_from_vfs(
// TODO: Is this even necessary anymore? // TODO: Is this even necessary anymore?
match file_name { match file_name {
"init.server.luau" | "init.server.lua" | "init.client.luau" | "init.client.lua" "init.server.luau" | "init.server.lua" | "init.client.luau" | "init.client.lua"
| "init.luau" | "init.lua" | "init.csv" => return Ok(None), | "init.plugin.luau" | "init.plugin.lua" | "init.luau" | "init.lua" | "init.csv" => {
return Ok(None)
}
_ => {} _ => {}
} }
@@ -124,6 +126,8 @@ fn get_dir_middleware<'path>(
(Middleware::ServerScriptDir, "init.server.lua"), (Middleware::ServerScriptDir, "init.server.lua"),
(Middleware::ClientScriptDir, "init.client.luau"), (Middleware::ClientScriptDir, "init.client.luau"),
(Middleware::ClientScriptDir, "init.client.lua"), (Middleware::ClientScriptDir, "init.client.lua"),
(Middleware::PluginScriptDir, "init.plugin.lua"),
(Middleware::PluginScriptDir, "init.plugin.luau"),
(Middleware::CsvDir, "init.csv"), (Middleware::CsvDir, "init.csv"),
] ]
}); });
@@ -205,6 +209,8 @@ pub enum Middleware {
#[serde(skip_deserializing)] #[serde(skip_deserializing)]
ClientScriptDir, ClientScriptDir,
#[serde(skip_deserializing)] #[serde(skip_deserializing)]
PluginScriptDir,
#[serde(skip_deserializing)]
ModuleScriptDir, ModuleScriptDir,
#[serde(skip_deserializing)] #[serde(skip_deserializing)]
CsvDir, CsvDir,
@@ -255,6 +261,9 @@ impl Middleware {
Self::ClientScriptDir => { Self::ClientScriptDir => {
snapshot_lua_init(context, vfs, path, name, ScriptType::Client) snapshot_lua_init(context, vfs, path, name, ScriptType::Client)
} }
Self::PluginScriptDir => {
snapshot_lua_init(context, vfs, path, name, ScriptType::Plugin)
}
Self::ModuleScriptDir => { Self::ModuleScriptDir => {
snapshot_lua_init(context, vfs, path, name, ScriptType::Module) snapshot_lua_init(context, vfs, path, name, ScriptType::Module)
} }
@@ -297,6 +306,7 @@ impl Middleware {
Middleware::Dir => syncback_dir(snapshot), Middleware::Dir => syncback_dir(snapshot),
Middleware::ServerScriptDir => syncback_lua_init(ScriptType::Server, snapshot), Middleware::ServerScriptDir => syncback_lua_init(ScriptType::Server, snapshot),
Middleware::ClientScriptDir => syncback_lua_init(ScriptType::Client, snapshot), Middleware::ClientScriptDir => syncback_lua_init(ScriptType::Client, snapshot),
Middleware::PluginScriptDir => syncback_lua_init(ScriptType::Plugin, snapshot),
Middleware::ModuleScriptDir => syncback_lua_init(ScriptType::Module, snapshot), Middleware::ModuleScriptDir => syncback_lua_init(ScriptType::Module, snapshot),
Middleware::CsvDir => syncback_csv_init(snapshot), Middleware::CsvDir => syncback_csv_init(snapshot),
@@ -318,6 +328,7 @@ impl Middleware {
Middleware::Dir Middleware::Dir
| Middleware::ServerScriptDir | Middleware::ServerScriptDir
| Middleware::ClientScriptDir | Middleware::ClientScriptDir
| Middleware::PluginScriptDir
| Middleware::ModuleScriptDir | Middleware::ModuleScriptDir
| Middleware::CsvDir | Middleware::CsvDir
) )

View File

@@ -344,6 +344,11 @@ pub fn syncback_project<'sync>(
let mut new_child_map = HashMap::new(); let mut new_child_map = HashMap::new();
let mut node_changed_map = Vec::new(); let mut node_changed_map = Vec::new();
// Tracks whether any stale default-valued properties were removed from
// project nodes. If so, we must reserialize even if
// project_node_should_reserialize wouldn't otherwise detect a change
// (it only compares node properties forward, not in reverse).
let mut removed_stale_properties = false;
let mut node_queue = VecDeque::with_capacity(1); let mut node_queue = VecDeque::with_capacity(1);
node_queue.push_back((&mut project.tree, old_inst, snapshot.new_inst())); node_queue.push_back((&mut project.tree, old_inst, snapshot.new_inst()));
@@ -402,10 +407,12 @@ pub fn syncback_project<'sync>(
// We only want to set properties if it needs it. // We only want to set properties if it needs it.
if !middleware.handles_own_properties() { if !middleware.handles_own_properties() {
project_node_property_syncback_path(snapshot, new_inst, node); removed_stale_properties |=
project_node_property_syncback_path(snapshot, new_inst, node);
} }
} else { } else {
project_node_property_syncback_no_path(snapshot, new_inst, node); removed_stale_properties |=
project_node_property_syncback_no_path(snapshot, new_inst, node);
} }
for child_ref in new_inst.children() { for child_ref in new_inst.children() {
@@ -507,12 +514,18 @@ pub fn syncback_project<'sync>(
} }
let mut fs_snapshot = FsSnapshot::new(); let mut fs_snapshot = FsSnapshot::new();
for (node_properties, node_attributes, old_inst) in node_changed_map { let mut needs_reserialize = removed_stale_properties;
if project_node_should_reserialize(node_properties, node_attributes, old_inst)? { if !needs_reserialize {
fs_snapshot.add_file(project_path, serde_json::to_vec_pretty(&project)?); for (node_properties, node_attributes, old_inst) in node_changed_map {
break; if project_node_should_reserialize(node_properties, node_attributes, old_inst)? {
needs_reserialize = true;
break;
}
} }
} }
if needs_reserialize {
fs_snapshot.add_file(project_path, serde_json::to_vec_pretty(&project)?);
}
Ok(SyncbackReturn { Ok(SyncbackReturn {
fs_snapshot, fs_snapshot,
@@ -521,15 +534,18 @@ pub fn syncback_project<'sync>(
}) })
} }
/// Syncs properties from the new instance into the project node.
/// Returns `true` if any stale properties were removed (i.e. properties
/// that existed in the project node but are now at their engine default).
fn project_node_property_syncback( fn project_node_property_syncback(
_snapshot: &SyncbackSnapshot, _snapshot: &SyncbackSnapshot,
filtered_properties: UstrMap<&Variant>, filtered_properties: UstrMap<&Variant>,
new_inst: &Instance, new_inst: &Instance,
node: &mut ProjectNode, node: &mut ProjectNode,
) { ) -> bool {
let properties = &mut node.properties; let properties = &mut node.properties;
let mut attributes = BTreeMap::new(); let mut attributes = BTreeMap::new();
for (name, value) in filtered_properties { for (&name, &value) in &filtered_properties {
match value { match value {
Variant::Attributes(attrs) => { Variant::Attributes(attrs) => {
for (attr_name, attr_value) in attrs.iter() { for (attr_name, attr_value) in attrs.iter() {
@@ -552,14 +568,48 @@ fn project_node_property_syncback(
} }
} }
} }
// Remove stale properties: entries that exist in the project node's
// $properties but are no longer in the filtered (non-default) properties
// from the instance. This handles the case where Studio resets a property
// to its engine default — filter_properties won't include it, so we need
// to clean up the now-stale project entry.
let class_data = rbx_reflection_database::get()
.ok()
.and_then(|db| db.classes.get(new_inst.class.as_str()));
let len_before = properties.len();
properties.retain(|prop_name, _| {
if filtered_properties.contains_key(prop_name) {
return true;
}
// Only remove if the property has a known default value in the
// reflection database. If there's no default, the property might be
// absent from the instance for other reasons (e.g. unknown property),
// so we conservatively keep it.
if let Some(data) = &class_data {
if data.default_properties.contains_key(prop_name.as_str()) {
log::debug!(
"Removing stale property '{}' from project node for class '{}': \
value has been reset to engine default",
prop_name,
new_inst.class
);
return false;
}
}
true
});
let removed_stale = properties.len() < len_before;
node.attributes = attributes; node.attributes = attributes;
removed_stale
} }
fn project_node_property_syncback_path( fn project_node_property_syncback_path(
snapshot: &SyncbackSnapshot, snapshot: &SyncbackSnapshot,
new_inst: &Instance, new_inst: &Instance,
node: &mut ProjectNode, node: &mut ProjectNode,
) { ) -> bool {
let filtered_properties = snapshot let filtered_properties = snapshot
.get_path_filtered_properties(new_inst.referent()) .get_path_filtered_properties(new_inst.referent())
.unwrap(); .unwrap();
@@ -570,7 +620,7 @@ fn project_node_property_syncback_no_path(
snapshot: &SyncbackSnapshot, snapshot: &SyncbackSnapshot,
new_inst: &Instance, new_inst: &Instance,
node: &mut ProjectNode, node: &mut ProjectNode,
) { ) -> bool {
let filtered_properties = filter_properties(snapshot.project(), new_inst); let filtered_properties = filter_properties(snapshot.project(), new_inst);
project_node_property_syncback(snapshot, filtered_properties, new_inst, node) project_node_property_syncback(snapshot, filtered_properties, new_inst, node)
} }

View File

@@ -28,7 +28,7 @@ pub fn snapshot_rbxm(
.metadata( .metadata(
InstanceMetadata::new() InstanceMetadata::new()
.instigating_source(path) .instigating_source(path)
.relevant_paths(vec![path.to_path_buf()]) .relevant_paths(vec![vfs.canonicalize(path)?])
.context(context), .context(context),
); );

View File

@@ -31,7 +31,7 @@ pub fn snapshot_rbxmx(
.metadata( .metadata(
InstanceMetadata::new() InstanceMetadata::new()
.instigating_source(path) .instigating_source(path)
.relevant_paths(vec![path.to_path_buf()]) .relevant_paths(vec![vfs.canonicalize(path)?])
.context(context), .context(context),
); );

View File

@@ -15,6 +15,8 @@ metadata:
- /root/init.server.luau - /root/init.server.luau
- /root/init.client.lua - /root/init.client.lua
- /root/init.client.luau - /root/init.client.luau
- /root/init.plugin.lua
- /root/init.plugin.luau
- /root/init.csv - /root/init.csv
- /root/init.meta.json - /root/init.meta.json
- /root/init.meta.jsonc - /root/init.meta.jsonc

View File

@@ -15,6 +15,8 @@ metadata:
- /root/init.server.luau - /root/init.server.luau
- /root/init.client.lua - /root/init.client.lua
- /root/init.client.luau - /root/init.client.luau
- /root/init.plugin.lua
- /root/init.plugin.luau
- /root/init.csv - /root/init.csv
- /root/init.meta.json - /root/init.meta.json
- /root/init.meta.jsonc - /root/init.meta.jsonc

View File

@@ -15,6 +15,8 @@ metadata:
- /foo/init.server.luau - /foo/init.server.luau
- /foo/init.client.lua - /foo/init.client.lua
- /foo/init.client.luau - /foo/init.client.luau
- /foo/init.plugin.lua
- /foo/init.plugin.luau
- /foo/init.csv - /foo/init.csv
- /foo/init.meta.json - /foo/init.meta.json
- /foo/init.meta.jsonc - /foo/init.meta.jsonc

View File

@@ -15,6 +15,8 @@ metadata:
- /foo/init.server.luau - /foo/init.server.luau
- /foo/init.client.lua - /foo/init.client.lua
- /foo/init.client.luau - /foo/init.client.luau
- /foo/init.plugin.lua
- /foo/init.plugin.luau
- /foo/init.csv - /foo/init.csv
- /foo/init.meta.json - /foo/init.meta.json
- /foo/init.meta.jsonc - /foo/init.meta.jsonc
@@ -40,6 +42,8 @@ children:
- /foo/Child/init.server.luau - /foo/Child/init.server.luau
- /foo/Child/init.client.lua - /foo/Child/init.client.lua
- /foo/Child/init.client.luau - /foo/Child/init.client.luau
- /foo/Child/init.plugin.lua
- /foo/Child/init.plugin.luau
- /foo/Child/init.csv - /foo/Child/init.csv
- /foo/Child/init.meta.json - /foo/Child/init.meta.json
- /foo/Child/init.meta.jsonc - /foo/Child/init.meta.jsonc

View File

@@ -15,6 +15,8 @@ metadata:
- /root/init.server.luau - /root/init.server.luau
- /root/init.client.lua - /root/init.client.lua
- /root/init.client.luau - /root/init.client.luau
- /root/init.plugin.lua
- /root/init.plugin.luau
- /root/init.csv - /root/init.csv
- /root/init.meta.json - /root/init.meta.json
- /root/init.meta.jsonc - /root/init.meta.jsonc

View File

@@ -15,6 +15,8 @@ metadata:
- /root/init.server.luau - /root/init.server.luau
- /root/init.client.lua - /root/init.client.lua
- /root/init.client.luau - /root/init.client.luau
- /root/init.plugin.lua
- /root/init.plugin.luau
- /root/init.csv - /root/init.csv
- /root/init.meta.json - /root/init.meta.json
- /root/init.meta.jsonc - /root/init.meta.jsonc

View File

@@ -31,7 +31,7 @@ pub fn snapshot_toml(
.metadata( .metadata(
InstanceMetadata::new() InstanceMetadata::new()
.instigating_source(path) .instigating_source(path)
.relevant_paths(vec![path.to_path_buf()]) .relevant_paths(vec![vfs.canonicalize(path)?])
.context(context), .context(context),
); );

View File

@@ -28,7 +28,7 @@ pub fn snapshot_txt(
.metadata( .metadata(
InstanceMetadata::new() InstanceMetadata::new()
.instigating_source(path) .instigating_source(path)
.relevant_paths(vec![path.to_path_buf()]) .relevant_paths(vec![vfs.canonicalize(path)?])
.context(context), .context(context),
); );

View File

@@ -37,7 +37,7 @@ pub fn snapshot_yaml(
.metadata( .metadata(
InstanceMetadata::new() InstanceMetadata::new()
.instigating_source(path) .instigating_source(path)
.relevant_paths(vec![path.to_path_buf()]) .relevant_paths(vec![vfs.canonicalize(path)?])
.context(context), .context(context),
); );

View File

@@ -35,6 +35,7 @@ pub fn name_for_inst<'old>(
| Middleware::CsvDir | Middleware::CsvDir
| Middleware::ServerScriptDir | Middleware::ServerScriptDir
| Middleware::ClientScriptDir | Middleware::ClientScriptDir
| Middleware::PluginScriptDir
| Middleware::ModuleScriptDir => Cow::Owned(new_inst.name.clone()), | Middleware::ModuleScriptDir => Cow::Owned(new_inst.name.clone()),
_ => { _ => {
let extension = extension_for_middleware(middleware); let extension = extension_for_middleware(middleware);
@@ -78,6 +79,7 @@ pub fn extension_for_middleware(middleware: Middleware) -> &'static str {
| Middleware::CsvDir | Middleware::CsvDir
| Middleware::ServerScriptDir | Middleware::ServerScriptDir
| Middleware::ClientScriptDir | Middleware::ClientScriptDir
| Middleware::PluginScriptDir
| Middleware::ModuleScriptDir => { | Middleware::ModuleScriptDir => {
unimplemented!("directory middleware requires special treatment") unimplemented!("directory middleware requires special treatment")
} }

View File

@@ -21,7 +21,7 @@ use std::{
}; };
use crate::{ use crate::{
glob::Glob, glob::{Glob, IgnorableGlob},
snapshot::{InstanceWithMeta, RojoTree}, snapshot::{InstanceWithMeta, RojoTree},
snapshot_middleware::Middleware, snapshot_middleware::Middleware,
syncback::ref_properties::{collect_referents, link_referents}, syncback::ref_properties::{collect_referents, link_referents},
@@ -301,6 +301,7 @@ pub fn get_best_middleware(snapshot: &SyncbackSnapshot) -> Middleware {
static JSON_MODEL_CLASSES: OnceLock<HashSet<&str>> = OnceLock::new(); static JSON_MODEL_CLASSES: OnceLock<HashSet<&str>> = OnceLock::new();
let json_model_classes = JSON_MODEL_CLASSES.get_or_init(|| { let json_model_classes = JSON_MODEL_CLASSES.get_or_init(|| {
[ [
"Actor",
"Sound", "Sound",
"SoundGroup", "SoundGroup",
"Sky", "Sky",
@@ -318,6 +319,11 @@ pub fn get_best_middleware(snapshot: &SyncbackSnapshot) -> Middleware {
"ChatInputBarConfiguration", "ChatInputBarConfiguration",
"BubbleChatConfiguration", "BubbleChatConfiguration",
"ChannelTabsConfiguration", "ChannelTabsConfiguration",
"RemoteEvent",
"UnreliableRemoteEvent",
"RemoteFunction",
"BindableEvent",
"BindableFunction",
] ]
.into() .into()
}); });
@@ -353,6 +359,7 @@ pub fn get_best_middleware(snapshot: &SyncbackSnapshot) -> Middleware {
middleware = match middleware { middleware = match middleware {
Middleware::ServerScript => Middleware::ServerScriptDir, Middleware::ServerScript => Middleware::ServerScriptDir,
Middleware::ClientScript => Middleware::ClientScriptDir, Middleware::ClientScript => Middleware::ClientScriptDir,
Middleware::PluginScript => Middleware::PluginScriptDir,
Middleware::ModuleScript => Middleware::ModuleScriptDir, Middleware::ModuleScript => Middleware::ModuleScriptDir,
Middleware::Csv => Middleware::CsvDir, Middleware::Csv => Middleware::CsvDir,
Middleware::JsonModel | Middleware::Text => Middleware::Dir, Middleware::JsonModel | Middleware::Text => Middleware::Dir,
@@ -408,18 +415,18 @@ pub struct SyncbackRules {
} }
impl SyncbackRules { impl SyncbackRules {
pub fn compile_globs(&self) -> anyhow::Result<Vec<Glob>> { pub fn compile_globs(&self) -> anyhow::Result<Vec<IgnorableGlob>> {
let mut globs = Vec::with_capacity(self.ignore_paths.len()); let mut globs = Vec::with_capacity(self.ignore_paths.len());
let dir_ignore_paths = self.create_ignore_dir_paths.unwrap_or(true); let dir_ignore_paths = self.create_ignore_dir_paths.unwrap_or(true);
for pattern in &self.ignore_paths { for pattern in &self.ignore_paths {
let glob = Glob::new(pattern) let glob = IgnorableGlob::new(pattern)
.with_context(|| format!("the pattern '{pattern}' is not a valid glob"))?; .with_context(|| format!("the pattern '{pattern}' is not a valid glob"))?;
globs.push(glob); globs.push(glob);
if dir_ignore_paths { if dir_ignore_paths {
if let Some(dir_pattern) = pattern.strip_suffix("/**") { if let Some(dir_pattern) = pattern.strip_suffix("/**") {
if let Ok(glob) = Glob::new(dir_pattern) { if let Ok(glob) = IgnorableGlob::new(dir_pattern) {
globs.push(glob) globs.push(glob)
} }
} }
@@ -430,7 +437,7 @@ impl SyncbackRules {
} }
} }
fn is_valid_path(globs: &Option<Vec<Glob>>, base_path: &Path, path: &Path) -> bool { fn is_valid_path(globs: &Option<Vec<IgnorableGlob>>, base_path: &Path, path: &Path) -> bool {
let git_glob = GIT_IGNORE_GLOB.get_or_init(|| Glob::new(".git/**").unwrap()); let git_glob = GIT_IGNORE_GLOB.get_or_init(|| Glob::new(".git/**").unwrap());
let test_path = match path.strip_prefix(base_path) { let test_path = match path.strip_prefix(base_path) {
Ok(suffix) => suffix, Ok(suffix) => suffix,
@@ -440,11 +447,16 @@ fn is_valid_path(globs: &Option<Vec<Glob>>, base_path: &Path, path: &Path) -> bo
return false; return false;
} }
if let Some(ref ignore_paths) = globs { if let Some(ref ignore_paths) = globs {
// Gitignore-style "last match wins"
let mut ignored = false;
for glob in ignore_paths { for glob in ignore_paths {
if glob.is_match(test_path) { if glob.is_match(test_path) {
return false; ignored = !glob.is_negation();
} }
} }
if ignored {
return false;
}
} }
true true
} }
@@ -532,3 +544,71 @@ fn strip_unknown_root_children(new: &mut WeakDom, old: &RojoTree) {
new.destroy(child_ref); new.destroy(child_ref);
} }
} }
#[cfg(test)]
mod test {
use super::*;
fn rules(ignore_paths: &[&str], create_ignore_dir_paths: Option<bool>) -> SyncbackRules {
SyncbackRules {
ignore_trees: Vec::new(),
ignore_paths: ignore_paths.iter().map(|s| s.to_string()).collect(),
ignore_properties: IndexMap::new(),
sync_current_camera: None,
sync_unscriptable: None,
ignore_referents: None,
create_ignore_dir_paths,
}
}
#[test]
fn ignore_paths_negation() {
let globs = Some(
rules(&["**/*.lua", "!keep.lua"], Some(false))
.compile_globs()
.unwrap(),
);
let base = Path::new("/test");
// A later negation re-includes a path matched by an earlier pattern.
assert!(!is_valid_path(&globs, base, Path::new("/test/foo.lua")));
assert!(is_valid_path(&globs, base, Path::new("/test/keep.lua")));
// Paths matched by no rule are valid.
assert!(is_valid_path(&globs, base, Path::new("/test/plain.txt")));
}
#[test]
fn ignore_paths_negation_with_dir_expansion() {
// With `create_ignore_dir_paths`, a negated `foo/**` pattern should also
// re-include the `foo` directory itself, mirroring the file rule.
let globs = Some(
rules(&["**/*", "!keep/**"], Some(true))
.compile_globs()
.unwrap(),
);
let base = Path::new("/test");
assert!(!is_valid_path(&globs, base, Path::new("/test/drop/a.lua")));
assert!(is_valid_path(&globs, base, Path::new("/test/keep")));
assert!(is_valid_path(&globs, base, Path::new("/test/keep/a.lua")));
}
#[test]
fn ignore_paths_escaped_bang_is_literal() {
// `\!literal.lua` should ignore a file literally named `!literal.lua`
// rather than being parsed as a negation.
let globs = Some(
rules(&[r"\!literal.lua"], Some(false))
.compile_globs()
.unwrap(),
);
let base = Path::new("/test");
assert!(!globs.as_ref().unwrap()[0].is_negation());
assert!(!is_valid_path(
&globs,
base,
Path::new("/test/!literal.lua")
));
}
}

View File

@@ -1,7 +1,7 @@
//! Defines Rojo's HTTP API, all under /api. These endpoints generally return //! Defines Rojo's HTTP API, all under /api. These endpoints generally return
//! JSON. //! JSON.
use std::{collections::HashMap, fs, path::PathBuf, str::FromStr, sync::Arc}; use std::{collections::HashMap, fs, net::SocketAddr, path::PathBuf, str::FromStr, sync::Arc};
use futures::{sink::SinkExt, stream::StreamExt}; use futures::{sink::SinkExt, stream::StreamExt};
use hyper::{body, Body, Method, Request, Response, StatusCode}; use hyper::{body, Body, Method, Request, Response, StatusCode};
@@ -13,7 +13,6 @@ use rbx_dom_weak::{
}; };
use crate::{ use crate::{
json,
serve_session::ServeSession, serve_session::ServeSession,
snapshot::{InstanceWithMeta, PatchSet, PatchUpdate}, snapshot::{InstanceWithMeta, PatchSet, PatchUpdate},
web::{ web::{
@@ -22,16 +21,20 @@ use crate::{
ServerInfoResponse, SocketPacket, SocketPacketBody, SocketPacketType, SubscribeMessage, ServerInfoResponse, SocketPacket, SocketPacketBody, SocketPacketType, SubscribeMessage,
WriteRequest, WriteResponse, PROTOCOL_VERSION, SERVER_VERSION, WriteRequest, WriteResponse, PROTOCOL_VERSION, SERVER_VERSION,
}, },
util::{json, json_ok}, origin::canonical,
util::{deserialize_msgpack, msgpack, msgpack_ok, serialize_msgpack},
}, },
web_api::{ web_api::{
BufferEncode, InstanceUpdate, RefPatchRequest, RefPatchResponse, SerializeRequest, InstanceUpdate, RefPatchRequest, RefPatchResponse, SerializeRequest, SerializeResponse,
SerializeResponse,
}, },
}; };
pub async fn call(serve_session: Arc<ServeSession>, mut request: Request<Body>) -> Response<Body> { pub async fn call(
let service = ApiService::new(serve_session); serve_session: Arc<ServeSession>,
remote_addr: SocketAddr,
mut request: Request<Body>,
) -> Response<Body> {
let service = ApiService::new(serve_session, remote_addr);
match (request.method(), request.uri().path()) { match (request.method(), request.uri().path()) {
(&Method::GET, "/api/rojo") => service.handle_api_rojo().await, (&Method::GET, "/api/rojo") => service.handle_api_rojo().await,
@@ -42,7 +45,7 @@ pub async fn call(serve_session: Arc<ServeSession>, mut request: Request<Body>)
if is_upgrade_request(&request) { if is_upgrade_request(&request) {
service.handle_api_socket(&mut request).await service.handle_api_socket(&mut request).await
} else { } else {
json( msgpack(
ErrorResponse::bad_request( ErrorResponse::bad_request(
"/api/socket must be called as a websocket upgrade request", "/api/socket must be called as a websocket upgrade request",
), ),
@@ -58,7 +61,7 @@ pub async fn call(serve_session: Arc<ServeSession>, mut request: Request<Body>)
} }
(&Method::POST, "/api/write") => service.handle_api_write(request).await, (&Method::POST, "/api/write") => service.handle_api_write(request).await,
(_method, path) => json( (_method, path) => msgpack(
ErrorResponse::not_found(format!("Route not found: {}", path)), ErrorResponse::not_found(format!("Route not found: {}", path)),
StatusCode::NOT_FOUND, StatusCode::NOT_FOUND,
), ),
@@ -67,11 +70,15 @@ pub async fn call(serve_session: Arc<ServeSession>, mut request: Request<Body>)
pub struct ApiService { pub struct ApiService {
serve_session: Arc<ServeSession>, serve_session: Arc<ServeSession>,
remote_addr: SocketAddr,
} }
impl ApiService { impl ApiService {
pub fn new(serve_session: Arc<ServeSession>) -> Self { pub fn new(serve_session: Arc<ServeSession>, remote_addr: SocketAddr) -> Self {
ApiService { serve_session } ApiService {
serve_session,
remote_addr,
}
} }
/// Get a summary of information about the server /// Get a summary of information about the server
@@ -79,7 +86,7 @@ impl ApiService {
let tree = self.serve_session.tree(); let tree = self.serve_session.tree();
let root_instance_id = tree.get_root_id(); let root_instance_id = tree.get_root_id();
json_ok(&ServerInfoResponse { msgpack_ok(&ServerInfoResponse {
server_version: SERVER_VERSION.to_owned(), server_version: SERVER_VERSION.to_owned(),
protocol_version: PROTOCOL_VERSION, protocol_version: PROTOCOL_VERSION,
session_id: self.serve_session.session_id(), session_id: self.serve_session.session_id(),
@@ -98,7 +105,7 @@ impl ApiService {
let input_cursor: u32 = match argument.parse() { let input_cursor: u32 = match argument.parse() {
Ok(v) => v, Ok(v) => v,
Err(err) => { Err(err) => {
return json( return msgpack(
ErrorResponse::bad_request(format!("Malformed message cursor: {}", err)), ErrorResponse::bad_request(format!("Malformed message cursor: {}", err)),
StatusCode::BAD_REQUEST, StatusCode::BAD_REQUEST,
); );
@@ -109,7 +116,7 @@ impl ApiService {
let (response, websocket) = match upgrade(request, None) { let (response, websocket) = match upgrade(request, None) {
Ok(result) => result, Ok(result) => result,
Err(err) => { Err(err) => {
return json( return msgpack(
ErrorResponse::internal_error(format!("WebSocket upgrade failed: {}", err)), ErrorResponse::internal_error(format!("WebSocket upgrade failed: {}", err)),
StatusCode::INTERNAL_SERVER_ERROR, StatusCode::INTERNAL_SERVER_ERROR,
); );
@@ -136,10 +143,10 @@ impl ApiService {
let body = body::to_bytes(request.into_body()).await.unwrap(); let body = body::to_bytes(request.into_body()).await.unwrap();
let request: WriteRequest = match json::from_slice(&body) { let request: WriteRequest = match deserialize_msgpack(&body) {
Ok(request) => request, Ok(request) => request,
Err(err) => { Err(err) => {
return json( return msgpack(
ErrorResponse::bad_request(format!("Invalid body: {}", err)), ErrorResponse::bad_request(format!("Invalid body: {}", err)),
StatusCode::BAD_REQUEST, StatusCode::BAD_REQUEST,
); );
@@ -147,7 +154,7 @@ impl ApiService {
}; };
if request.session_id != session_id { if request.session_id != session_id {
return json( return msgpack(
ErrorResponse::bad_request("Wrong session ID"), ErrorResponse::bad_request("Wrong session ID"),
StatusCode::BAD_REQUEST, StatusCode::BAD_REQUEST,
); );
@@ -173,7 +180,7 @@ impl ApiService {
}) })
.unwrap(); .unwrap();
json_ok(WriteResponse { session_id }) msgpack_ok(WriteResponse { session_id })
} }
async fn handle_api_read(&self, request: Request<Body>) -> Response<Body> { async fn handle_api_read(&self, request: Request<Body>) -> Response<Body> {
@@ -183,7 +190,7 @@ impl ApiService {
let requested_ids = match requested_ids { let requested_ids = match requested_ids {
Ok(ids) => ids, Ok(ids) => ids,
Err(_) => { Err(_) => {
return json( return msgpack(
ErrorResponse::bad_request("Malformed ID list"), ErrorResponse::bad_request("Malformed ID list"),
StatusCode::BAD_REQUEST, StatusCode::BAD_REQUEST,
); );
@@ -207,7 +214,7 @@ impl ApiService {
} }
} }
json_ok(ReadResponse { msgpack_ok(ReadResponse {
session_id: self.serve_session.session_id(), session_id: self.serve_session.session_id(),
message_cursor, message_cursor,
instances, instances,
@@ -225,10 +232,10 @@ impl ApiService {
let session_id = self.serve_session.session_id(); let session_id = self.serve_session.session_id();
let body = body::to_bytes(request.into_body()).await.unwrap(); let body = body::to_bytes(request.into_body()).await.unwrap();
let request: SerializeRequest = match json::from_slice(&body) { let request: SerializeRequest = match deserialize_msgpack(&body) {
Ok(request) => request, Ok(request) => request,
Err(err) => { Err(err) => {
return json( return msgpack(
ErrorResponse::bad_request(format!("Invalid body: {}", err)), ErrorResponse::bad_request(format!("Invalid body: {}", err)),
StatusCode::BAD_REQUEST, StatusCode::BAD_REQUEST,
); );
@@ -236,7 +243,7 @@ impl ApiService {
}; };
if request.session_id != session_id { if request.session_id != session_id {
return json( return msgpack(
ErrorResponse::bad_request("Wrong session ID"), ErrorResponse::bad_request("Wrong session ID"),
StatusCode::BAD_REQUEST, StatusCode::BAD_REQUEST,
); );
@@ -269,7 +276,7 @@ impl ApiService {
response_dom.transfer_within(child_ref, object_value); response_dom.transfer_within(child_ref, object_value);
} else { } else {
json( return msgpack(
ErrorResponse::bad_request(format!("provided id {id} is not in the tree")), ErrorResponse::bad_request(format!("provided id {id} is not in the tree")),
StatusCode::BAD_REQUEST, StatusCode::BAD_REQUEST,
); );
@@ -280,9 +287,9 @@ impl ApiService {
let mut source = Vec::new(); let mut source = Vec::new();
rbx_binary::to_writer(&mut source, &response_dom, &[response_dom.root_ref()]).unwrap(); rbx_binary::to_writer(&mut source, &response_dom, &[response_dom.root_ref()]).unwrap();
json_ok(SerializeResponse { msgpack_ok(SerializeResponse {
session_id: self.serve_session.session_id(), session_id: self.serve_session.session_id(),
model_contents: BufferEncode::new(source), model_contents: source,
}) })
} }
@@ -294,10 +301,10 @@ impl ApiService {
let session_id = self.serve_session.session_id(); let session_id = self.serve_session.session_id();
let body = body::to_bytes(request.into_body()).await.unwrap(); let body = body::to_bytes(request.into_body()).await.unwrap();
let request: RefPatchRequest = match json::from_slice(&body) { let request: RefPatchRequest = match deserialize_msgpack(&body) {
Ok(request) => request, Ok(request) => request,
Err(err) => { Err(err) => {
return json( return msgpack(
ErrorResponse::bad_request(format!("Invalid body: {}", err)), ErrorResponse::bad_request(format!("Invalid body: {}", err)),
StatusCode::BAD_REQUEST, StatusCode::BAD_REQUEST,
); );
@@ -305,7 +312,7 @@ impl ApiService {
}; };
if request.session_id != session_id { if request.session_id != session_id {
return json( return msgpack(
ErrorResponse::bad_request("Wrong session ID"), ErrorResponse::bad_request("Wrong session ID"),
StatusCode::BAD_REQUEST, StatusCode::BAD_REQUEST,
); );
@@ -338,7 +345,7 @@ impl ApiService {
} }
} }
json_ok(RefPatchResponse { msgpack_ok(RefPatchResponse {
session_id: self.serve_session.session_id(), session_id: self.serve_session.session_id(),
patch: SubscribeMessage { patch: SubscribeMessage {
added: HashMap::new(), added: HashMap::new(),
@@ -350,11 +357,35 @@ impl ApiService {
/// Open a script with the given ID in the user's default text editor. /// Open a script with the given ID in the user's default text editor.
async fn handle_api_open(&self, request: Request<Body>) -> Response<Body> { async fn handle_api_open(&self, request: Request<Body>) -> Response<Body> {
// Opening a file launches a local program, so it must never be reachable
// by a remote client even when the server is bound to an exposed address.
//
// `remote_addr` is the immediate peer, which is the best locality signal
// we have: the legitimate caller is a sandboxed Roblox plugin whose only
// credential is being able to reach the port, so there is no secret to
// authenticate it with. A connection forwarded over loopback by an
// SSH/Tailscale tunnel or a local reverse proxy therefore appears local
// and is allowed. That is delegated trust rather than a bypass: by
// standing up that tunnel or proxy the user has decided the remote end is
// trusted, and reachability is bounded by that hop's own authentication
// (e.g. SSH keys or Tailscale ACLs). This gate only stops direct,
// unauthenticated peers.
//
// An IPv4 client reaching a dual-stack (`::`) bind appears as an
// IPv4-mapped IPv6 peer (`::ffff:127.0.0.1`), so canonicalize to the bare
// IPv4 form before the loopback test, matching `origin`'s handling.
if !canonical(self.remote_addr.ip()).is_loopback() {
return msgpack(
ErrorResponse::forbidden("/api/open is only available to local clients"),
StatusCode::FORBIDDEN,
);
}
let argument = &request.uri().path()["/api/open/".len()..]; let argument = &request.uri().path()["/api/open/".len()..];
let requested_id = match Ref::from_str(argument) { let requested_id = match Ref::from_str(argument) {
Ok(id) => id, Ok(id) => id,
Err(_) => { Err(_) => {
return json( return msgpack(
ErrorResponse::bad_request("Invalid instance ID"), ErrorResponse::bad_request("Invalid instance ID"),
StatusCode::BAD_REQUEST, StatusCode::BAD_REQUEST,
); );
@@ -366,7 +397,7 @@ impl ApiService {
let instance = match tree.get_instance(requested_id) { let instance = match tree.get_instance(requested_id) {
Some(instance) => instance, Some(instance) => instance,
None => { None => {
return json( return msgpack(
ErrorResponse::bad_request("Instance not found"), ErrorResponse::bad_request("Instance not found"),
StatusCode::NOT_FOUND, StatusCode::NOT_FOUND,
); );
@@ -376,7 +407,7 @@ impl ApiService {
let script_path = match pick_script_path(instance) { let script_path = match pick_script_path(instance) {
Some(path) => path, Some(path) => path,
None => { None => {
return json( return msgpack(
ErrorResponse::bad_request( ErrorResponse::bad_request(
"No appropriate file could be found to open this script", "No appropriate file could be found to open this script",
), ),
@@ -389,7 +420,7 @@ impl ApiService {
Ok(()) => {} Ok(()) => {}
Err(error) => match error { Err(error) => match error {
OpenError::Io(io_error) => { OpenError::Io(io_error) => {
return json( return msgpack(
ErrorResponse::internal_error(format!( ErrorResponse::internal_error(format!(
"Attempting to open {} failed because of the following io error: {}", "Attempting to open {} failed because of the following io error: {}",
script_path.display(), script_path.display(),
@@ -403,7 +434,7 @@ impl ApiService {
status, status,
stderr, stderr,
} => { } => {
return json( return msgpack(
ErrorResponse::internal_error(format!( ErrorResponse::internal_error(format!(
r#"The command '{}' to open '{}' failed with the error code '{}'. r#"The command '{}' to open '{}' failed with the error code '{}'.
Error logs: Error logs:
@@ -419,7 +450,7 @@ impl ApiService {
}, },
}; };
json_ok(OpenResponse { msgpack_ok(OpenResponse {
session_id: self.serve_session.session_id(), session_id: self.serve_session.session_id(),
}) })
} }
@@ -483,7 +514,7 @@ async fn handle_websocket_subscription(
match result { match result {
Ok((new_cursor, messages)) => { Ok((new_cursor, messages)) => {
if !messages.is_empty() { if !messages.is_empty() {
let json_message = { let msgpack_message = {
let tree = tree_handle.lock().unwrap(); let tree = tree_handle.lock().unwrap();
let api_messages = messages let api_messages = messages
.into_iter() .into_iter()
@@ -499,12 +530,12 @@ async fn handle_websocket_subscription(
}), }),
}; };
serde_json::to_string(&response)? serialize_msgpack(response)?
}; };
log::debug!("Sending batch of messages over WebSocket subscription"); log::debug!("Sending batch of messages over WebSocket subscription");
if websocket.send(Message::Text(json_message)).await.is_err() { if websocket.send(Message::Binary(msgpack_message)).await.is_err() {
// Client disconnected // Client disconnected
log::debug!("WebSocket subscription closed by client"); log::debug!("WebSocket subscription closed by client");
break; break;

View File

@@ -249,31 +249,8 @@ pub struct SerializeRequest {
#[serde(rename_all = "camelCase")] #[serde(rename_all = "camelCase")]
pub struct SerializeResponse { pub struct SerializeResponse {
pub session_id: SessionId, pub session_id: SessionId,
pub model_contents: BufferEncode, #[serde(with = "serde_bytes")]
} pub model_contents: Vec<u8>,
/// Using this struct we can force Roblox to JSONDecode this as a buffer.
/// This is what Roblox's serde APIs use, so it saves a step in the plugin.
#[derive(Debug, Serialize, Deserialize)]
pub struct BufferEncode {
m: (),
t: Cow<'static, str>,
base64: String,
}
impl BufferEncode {
pub fn new(content: Vec<u8>) -> Self {
let base64 = data_encoding::BASE64.encode(&content);
Self {
m: (),
t: Cow::Borrowed("buffer"),
base64,
}
}
pub fn model(&self) -> &str {
&self.base64
}
} }
#[derive(Debug, Serialize, Deserialize)] #[derive(Debug, Serialize, Deserialize)]
@@ -313,6 +290,13 @@ impl ErrorResponse {
} }
} }
pub fn forbidden<S: Into<String>>(details: S) -> Self {
Self {
kind: ErrorResponseKind::Forbidden,
details: details.into(),
}
}
pub fn internal_error<S: Into<String>>(details: S) -> Self { pub fn internal_error<S: Into<String>>(details: S) -> Self {
Self { Self {
kind: ErrorResponseKind::InternalError, kind: ErrorResponseKind::InternalError,
@@ -325,5 +309,6 @@ impl ErrorResponse {
pub enum ErrorResponseKind { pub enum ErrorResponseKind {
NotFound, NotFound,
BadRequest, BadRequest,
Forbidden,
InternalError, InternalError,
} }

View File

@@ -5,6 +5,7 @@
mod api; mod api;
mod assets; mod assets;
pub mod interface; pub mod interface;
mod origin;
mod ui; mod ui;
mod util; mod util;
@@ -12,8 +13,9 @@ use std::convert::Infallible;
use std::net::SocketAddr; use std::net::SocketAddr;
use std::sync::Arc; use std::sync::Arc;
use anyhow::Context;
use hyper::{ use hyper::{
server::Server, server::{conn::AddrStream, Server},
service::{make_service_fn, service_fn}, service::{make_service_fn, service_fn},
Body, Request, Body, Request,
}; };
@@ -30,19 +32,44 @@ impl LiveServer {
LiveServer { serve_session } LiveServer { serve_session }
} }
pub fn start(self, address: SocketAddr) { /// Starts the server on the given address, blocking until it stops.
///
/// `allowed_hosts` are extra `Host`/`Origin` values to accept in addition to
/// localhost and the bind address (see [`origin::allowed_hosts`]).
///
/// `on_listening` is invoked once the server has successfully bound to the
/// address, so callers can defer printing any "listening" message until
/// after binding can no longer fail (e.g. due to the port being in use).
pub fn start(
self,
address: SocketAddr,
allowed_hosts: Vec<String>,
on_listening: impl FnOnce(),
) -> anyhow::Result<()> {
let serve_session = Arc::clone(&self.serve_session); let serve_session = Arc::clone(&self.serve_session);
let allowed_hosts = origin::allowed_hosts(address.ip(), address.port(), &allowed_hosts);
let make_service = make_service_fn(move |_conn| { let make_service = make_service_fn(move |conn: &AddrStream| {
let serve_session = Arc::clone(&serve_session); let serve_session = Arc::clone(&serve_session);
let allowed_hosts = allowed_hosts.clone();
let remote_addr = conn.remote_addr();
async { async move {
let service = move |req: Request<Body>| { let service = move |req: Request<Body>| {
let serve_session = Arc::clone(&serve_session); let serve_session = Arc::clone(&serve_session);
let allowed_hosts = allowed_hosts.clone();
async move { async move {
// Reject cross-origin requests before doing any work, to
// defend the local server against DNS rebinding.
if let Some(response) =
origin::check_request_origin(&req, allowed_hosts.as_ref())
{
return Ok::<_, Infallible>(response);
}
if req.uri().path().starts_with("/api") { if req.uri().path().starts_with("/api") {
Ok::<_, Infallible>(api::call(serve_session, req).await) Ok::<_, Infallible>(api::call(serve_session, remote_addr, req).await)
} else { } else {
Ok::<_, Infallible>(ui::call(serve_session, req).await) Ok::<_, Infallible>(ui::call(serve_session, req).await)
} }
@@ -53,9 +80,25 @@ impl LiveServer {
} }
}); });
let rt = Runtime::new().unwrap(); let rt = Runtime::new().context("Failed to start the async runtime for the web server")?;
let _guard = rt.enter(); let _guard = rt.enter();
let server = Server::bind(&address).serve(make_service); let server = Server::try_bind(&address)
rt.block_on(server).unwrap(); .with_context(|| {
format!(
"Could not start the Rojo server on {address}.\n\
The address may already be in use or reserved. Another Rojo server might already \
be running, or another program may be using that port.\n\
You can pick a different port with the --port option."
)
})?
.serve(make_service);
// Binding succeeded, so it's now safe to tell the user we're listening.
on_listening();
rt.block_on(server)
.context("The Rojo web server encountered a fatal error")?;
Ok(())
} }
} }

589
src/web/origin.rs Normal file
View File

@@ -0,0 +1,589 @@
//! Host/Origin validation used to defend the local Rojo server against DNS
//! rebinding attacks.
//!
//! When Rojo is bound to a loopback address (the default), a malicious web page
//! the developer visits cannot read the API responses directly because of the
//! browser Same-Origin Policy. However, a DNS rebinding attack can defeat that:
//! the page points its own hostname at `127.0.0.1` after loading, so the browser
//! treats requests to the Rojo server as same-origin. Validating that the `Host`
//! (and, if present, `Origin`) header refers to an address we recognize blocks
//! this, because the rebound request still carries the attacker's hostname, which
//! is a domain name rather than one of the IP literals we accept.
//!
//! Enforcement covers two kinds of bind:
//!
//! * Loopback (the default): only `localhost` and loopback literals are
//! accepted.
//! * A specific private/LAN address: `localhost`, loopback literals, and that exact bind
//! IP are accepted. Because the defense works by rejecting any `Host` that
//! isn't a recognized IP literal, clients must connect to a private bind by
//! IP. A hostname (e.g. `mypc.local`) is indistinguishable from an attacker
//! domain and is rejected.
//!
//! Enforcement is disabled for unspecified (`0.0.0.0`/`::`) and public binds: the
//! user has asked for broad, possibly-public exposure where arbitrary hostnames
//! may legitimately resolve to the server, so we can't build a meaningful
//! allowlist. Those binds get a startup warning instead. Two consequences worth
//! being explicit about: such a bind has no rebinding protection even for a
//! browser on the same machine; and even on a protected private bind this check
//! does nothing against a hostile peer already on the LAN, who can reach the
//! unauthenticated API directly. Both are the network-exposure risk the startup
//! warning addresses, not something the `Host` check is meant to cover.
//!
//! The allowlist can be widened with extra hosts (the `--allowed-hosts` CLI
//! option or a project's `serveAllowedHosts`), for example a hostname like
//! `mypc.lan` for reaching a network-exposed server by name. Listing any extra
//! host also turns enforcement back on for an unspecified or public bind that
//! would otherwise disable it, restricting that bind to localhost, the bind IP,
//! and the listed hosts.
use std::net::IpAddr;
use hyper::{
header::{HOST, ORIGIN},
http::uri::{Authority, Uri},
Body, Request, Response, StatusCode,
};
use crate::web::util::response;
/// The set of `Host`/`Origin` values accepted while enforcement is active.
#[derive(Debug, Clone)]
pub struct AllowedHosts {
port: u16,
/// The bind IP accepted in addition to `localhost`/loopback, set only for a
/// private (LAN) bind or a specific public bind that has extra hosts. `None`
/// for a loopback or unspecified bind, which has no single address to add.
bind_ip: Option<IpAddr>,
/// Extra `Host`/`Origin` values the user explicitly allowed, via the
/// `--allowed-hosts` option or a project's `serveAllowedHosts`. These are
/// hostnames such as `mypc.lan`, or IP literals, accepted in addition to
/// localhost and the bind IP. Each entry is already passed through
/// [`normalize_host`].
extra_hosts: Vec<String>,
}
impl AllowedHosts {
/// Returns whether the given host and optional port are allowed. The host is
/// accepted if it is `localhost`, a loopback IP literal, (on a private or
/// specific public bind) the exact bind IP, or one of the explicitly allowed
/// extra hosts. A request with no explicit port is accepted (the host
/// already has to be one we recognize).
fn allows(&self, host: &str, port: Option<u16>) -> bool {
let host = normalize_host(host);
let host_ok = host.eq_ignore_ascii_case("localhost")
|| host
.parse::<IpAddr>()
.ok()
.map(canonical)
.is_some_and(|ip| ip.is_loopback() || self.bind_ip == Some(ip))
|| self.allows_extra(host);
host_ok && port.is_none_or(|port| port == self.port)
}
/// Returns whether `host` matches one of the explicitly allowed extra hosts.
/// Entries that are IP literals are compared as addresses, so equivalent
/// forms (such as an IPv4-mapped IPv6 literal) match; everything else is
/// compared as a case-insensitive hostname.
fn allows_extra(&self, host: &str) -> bool {
let host_ip = host.parse::<IpAddr>().ok().map(canonical);
self.extra_hosts.iter().any(|allowed| {
match (host_ip, allowed.parse::<IpAddr>().map(canonical)) {
(Some(host_ip), Ok(allowed_ip)) => host_ip == allowed_ip,
_ => host.eq_ignore_ascii_case(allowed),
}
})
}
}
/// Builds the allowlist for a given bind address and any extra allowed hosts.
/// Returns `None` (validation disabled) when bound to an unspecified
/// (`0.0.0.0`/`::`) or public address and no extra hosts were given, where
/// arbitrary hostnames may legitimately resolve to the server. Listing extra
/// hosts keeps validation on even for those binds.
pub fn allowed_hosts(bind: IpAddr, port: u16, extra: &[String]) -> Option<AllowedHosts> {
let extra_hosts: Vec<String> = extra
.iter()
.map(|host| normalize_host(host.trim()).to_owned())
.filter(|host| !host.is_empty())
.collect();
if bind.is_loopback() {
Some(AllowedHosts {
port,
bind_ip: None,
extra_hosts,
})
} else if is_private_bind(bind) {
Some(AllowedHosts {
port,
bind_ip: Some(canonical(bind)),
extra_hosts,
})
} else if !extra_hosts.is_empty() {
// The bind is unspecified or public, where validation is normally
// disabled. By listing explicit hosts the user has opted back into it,
// so we accept localhost, the bind IP (when it is a specific address),
// and those hosts. An unspecified bind (`0.0.0.0`/`::`) has no single
// address to add.
let bind_ip = (!bind.is_unspecified()).then(|| canonical(bind));
Some(AllowedHosts {
port,
bind_ip,
extra_hosts,
})
} else {
None
}
}
/// Collapses an IPv4-mapped IPv6 address (`::ffff:192.168.0.1`) to its IPv4 form
/// so it classifies and compares consistently with the bare IPv4 address. Shared
/// with the `/api/open` peer check so it recognizes a mapped loopback peer too.
pub(crate) fn canonical(ip: IpAddr) -> IpAddr {
match ip {
IpAddr::V6(v6) => v6.to_ipv4_mapped().map(IpAddr::V4).unwrap_or(ip),
v4 => v4,
}
}
/// Returns whether a bind address is a specific private/link-local address, the
/// case where enforcement stays on with the bind IP added to the allowlist.
/// Unspecified, loopback, and public addresses are excluded (loopback is handled
/// separately by [`allowed_hosts`]).
fn is_private_bind(ip: IpAddr) -> bool {
match canonical(ip) {
IpAddr::V4(v4) => v4.is_private() || v4.is_link_local(),
IpAddr::V6(v6) => {
let first = v6.segments()[0];
// Unique local (fc00::/7) or link-local (fe80::/10). The std methods
// for these are still nightly-only, so test the prefix directly.
(first & 0xfe00) == 0xfc00 || (first & 0xffc0) == 0xfe80
}
}
}
/// Validates the `Host` and `Origin` headers of an incoming request against the
/// allowlist. Returns `Some` with a ready-to-send `404` response when the
/// request should be rejected, or `None` when it is allowed to proceed. When
/// `allowed` is `None` (unspecified or public bind) every request is accepted.
pub fn check_request_origin(
request: &Request<Body>,
allowed: Option<&AllowedHosts>,
) -> Option<Response<Body>> {
let allowed = allowed?;
// The Host header is mandatory and must refer to a local address.
let host_ok = request
.headers()
.get(HOST)
.and_then(|value| value.to_str().ok())
.and_then(parse_authority)
.is_some_and(|(host, port)| allowed.allows(&host, port));
if !host_ok {
return Some(reject());
}
// The Origin header is optional: non-browser clients such as the Roblox
// plugin never send it. When it is present (i.e. a browser made the request)
// it must also be local, which rejects a rebound page whose origin is still
// its own non-local hostname.
if let Some(origin) = request.headers().get(ORIGIN) {
let origin_ok = origin
.to_str()
.ok()
.and_then(parse_origin)
.is_some_and(|(host, port)| allowed.allows(&host, port));
if !origin_ok {
return Some(reject());
}
}
None
}
/// Normalizes a host literal for parsing/comparison: strips the surrounding
/// brackets from an IPv6 literal (e.g. `[::1]`) and drops any IPv6 zone id (e.g.
/// `fe80::1%eth0`), which `Ipv6Addr::from_str` would otherwise reject.
fn normalize_host(host: &str) -> &str {
let host = host
.strip_prefix('[')
.and_then(|host| host.strip_suffix(']'))
.unwrap_or(host);
host.split('%').next().unwrap_or(host)
}
/// Parses a `Host` header value (an authority such as `localhost:34872`) into
/// its host and optional port. Returns `None` if the authority carries userinfo
/// (e.g. `evil.com@localhost`), so a value whose host looks local only after the
/// userinfo is stripped can never sneak past the allowlist.
fn parse_authority(value: &str) -> Option<(String, Option<u16>)> {
let authority: Authority = value.parse().ok()?;
reject_userinfo(&authority)?;
Some((authority.host().to_owned(), authority.port_u16()))
}
/// Parses an `Origin` header value (an absolute URI such as
/// `http://localhost:34872`) into its host and optional port. Returns `None` for
/// origins without a host, such as the opaque `null` origin, or for origins whose
/// authority carries userinfo (see [`parse_authority`]).
fn parse_origin(value: &str) -> Option<(String, Option<u16>)> {
let uri: Uri = value.parse().ok()?;
reject_userinfo(uri.authority()?)?;
Some((uri.host()?.to_owned(), uri.port_u16()))
}
/// Returns `None` (rejecting the value) when an authority contains a userinfo
/// component, identified by the `@` separator. A bare host or `host:port` never
/// contains `@`, so this only fires on `userinfo@host` forms.
fn reject_userinfo(authority: &Authority) -> Option<()> {
if authority.as_str().contains('@') {
None
} else {
Some(())
}
}
/// Builds the response sent when a request fails Host/Origin validation. It is a
/// generic `404` with no Rojo-identifying body: a rejected request may be a
/// prober (or a DNS-rebound page's same-origin script, which could read the
/// body), so we reveal nothing rather than confirming a Rojo server is here.
fn reject() -> Response<Body> {
response(StatusCode::NOT_FOUND, "text/plain", "Not Found")
}
#[cfg(test)]
mod test {
use super::*;
use std::net::{Ipv4Addr, Ipv6Addr};
const PORT: u16 = 34872;
fn loopback_allowlist() -> Option<AllowedHosts> {
allowed_hosts(IpAddr::V4(Ipv4Addr::LOCALHOST), PORT, &[])
}
/// An allowlist for a server bound to a specific private (LAN) address.
fn private_allowlist() -> Option<AllowedHosts> {
allowed_hosts(IpAddr::V4(Ipv4Addr::new(192, 168, 1, 5)), PORT, &[])
}
/// An allowlist for `bind` with the given extra allowed hosts.
fn allowlist_with_hosts(bind: IpAddr, hosts: &[&str]) -> Option<AllowedHosts> {
let hosts: Vec<String> = hosts.iter().map(|host| host.to_string()).collect();
allowed_hosts(bind, PORT, &hosts)
}
fn request_with(headers: &[(&'static str, &str)]) -> Request<Body> {
let mut builder = Request::builder().uri("/api/rojo");
for (name, value) in headers {
builder = builder.header(*name, *value);
}
builder.body(Body::empty()).unwrap()
}
#[test]
fn loopback_bind_enables_enforcement() {
assert!(allowed_hosts(IpAddr::V4(Ipv4Addr::LOCALHOST), PORT, &[]).is_some());
assert!(allowed_hosts(IpAddr::V6(Ipv6Addr::LOCALHOST), PORT, &[]).is_some());
}
#[test]
fn private_bind_enables_enforcement() {
for bind in [
IpAddr::V4(Ipv4Addr::new(192, 168, 1, 5)), // 192.168.0.0/16
IpAddr::V4(Ipv4Addr::new(10, 0, 0, 1)), // 10.0.0.0/8
IpAddr::V4(Ipv4Addr::new(172, 16, 0, 1)), // 172.16.0.0/12
IpAddr::V4(Ipv4Addr::new(169, 254, 0, 1)), // link-local
IpAddr::V6(Ipv6Addr::new(0xfc00, 0, 0, 0, 0, 0, 0, 1)), // unique local
IpAddr::V6(Ipv6Addr::new(0xfe80, 0, 0, 0, 0, 0, 0, 1)), // link-local
] {
assert!(
allowed_hosts(bind, PORT, &[]).is_some(),
"private bind {bind} should enable enforcement"
);
}
}
#[test]
fn unspecified_or_public_bind_disables_enforcement() {
for bind in [
IpAddr::V4(Ipv4Addr::UNSPECIFIED), // 0.0.0.0
IpAddr::V6(Ipv6Addr::UNSPECIFIED), // ::
IpAddr::V4(Ipv4Addr::new(8, 8, 8, 8)), // public
IpAddr::V6(Ipv6Addr::new(0x2001, 0x4860, 0, 0, 0, 0, 0, 0x8888)), // public
] {
assert!(
allowed_hosts(bind, PORT, &[]).is_none(),
"bind {bind} should disable enforcement"
);
}
}
#[test]
fn accepts_local_hosts() {
let allowed = loopback_allowlist();
for host in [
format!("localhost:{PORT}"),
format!("127.0.0.1:{PORT}"),
format!("[::1]:{PORT}"),
"localhost".to_owned(),
] {
let request = request_with(&[("host", &host)]);
assert!(
check_request_origin(&request, allowed.as_ref()).is_none(),
"host {host} should be allowed"
);
}
}
#[test]
fn rejects_foreign_host() {
let allowed = loopback_allowlist();
let request = request_with(&[("host", "evil.com")]);
assert!(check_request_origin(&request, allowed.as_ref()).is_some());
}
#[test]
fn rejects_wrong_port() {
let allowed = loopback_allowlist();
let request = request_with(&[("host", "localhost:1234")]);
assert!(check_request_origin(&request, allowed.as_ref()).is_some());
}
#[test]
fn rejects_missing_host() {
let allowed = loopback_allowlist();
let request = request_with(&[]);
assert!(check_request_origin(&request, allowed.as_ref()).is_some());
}
#[test]
fn rejects_host_with_userinfo() {
let allowed = loopback_allowlist();
// The host parses to `localhost`, but the userinfo prefix must keep it
// from being treated as a local address.
let request = request_with(&[("host", &format!("evil.com@localhost:{PORT}"))]);
assert!(check_request_origin(&request, allowed.as_ref()).is_some());
}
#[test]
fn rejects_origin_with_userinfo() {
let allowed = loopback_allowlist();
let request = request_with(&[
("host", &format!("localhost:{PORT}")),
("origin", &format!("http://evil.com@localhost:{PORT}")),
]);
assert!(check_request_origin(&request, allowed.as_ref()).is_some());
}
#[test]
fn rejects_foreign_origin_even_with_local_host() {
let allowed = loopback_allowlist();
let request = request_with(&[
("host", &format!("localhost:{PORT}")),
("origin", &format!("http://evil.com:{PORT}")),
]);
assert!(check_request_origin(&request, allowed.as_ref()).is_some());
}
#[test]
fn rejects_null_origin() {
let allowed = loopback_allowlist();
let request = request_with(&[("host", &format!("localhost:{PORT}")), ("origin", "null")]);
assert!(check_request_origin(&request, allowed.as_ref()).is_some());
}
#[test]
fn accepts_local_origin() {
let allowed = loopback_allowlist();
let request = request_with(&[
("host", &format!("localhost:{PORT}")),
("origin", &format!("http://localhost:{PORT}")),
]);
assert!(check_request_origin(&request, allowed.as_ref()).is_none());
}
#[test]
fn private_bind_accepts_local_and_bind_ip_hosts() {
let allowed = private_allowlist();
for host in [
format!("192.168.1.5:{PORT}"),
format!("localhost:{PORT}"),
format!("127.0.0.1:{PORT}"),
format!("[::1]:{PORT}"),
"192.168.1.5".to_owned(),
] {
let request = request_with(&[("host", &host)]);
assert!(
check_request_origin(&request, allowed.as_ref()).is_none(),
"host {host} should be allowed on a private bind"
);
}
}
#[test]
fn private_bind_rejects_other_hosts() {
let allowed = private_allowlist();
for host in [
"evil.com", // a rebound attacker domain
"192.168.1.6", // a different private IP
"8.8.8.8", // a public IP
] {
let request = request_with(&[("host", host)]);
assert!(
check_request_origin(&request, allowed.as_ref()).is_some(),
"host {host} should be rejected on a private bind"
);
}
}
#[test]
fn private_bind_keeps_origin_strict() {
// A different private IP as Origin must be rejected even though the Host
// is the valid bind IP: the Origin check is not widened to arbitrary
// private addresses.
let allowed = private_allowlist();
let request = request_with(&[
("host", &format!("192.168.1.5:{PORT}")),
("origin", &format!("http://192.168.1.6:{PORT}")),
]);
assert!(check_request_origin(&request, allowed.as_ref()).is_some());
}
#[test]
fn private_bind_accepts_bind_ip_origin() {
let allowed = private_allowlist();
let request = request_with(&[
("host", &format!("192.168.1.5:{PORT}")),
("origin", &format!("http://192.168.1.5:{PORT}")),
]);
assert!(check_request_origin(&request, allowed.as_ref()).is_none());
}
#[test]
fn accepts_ipv4_mapped_bind_ip() {
// `::ffff:192.168.1.5` is the IPv4-mapped form of the bind IP and must
// be treated as equal to it.
let allowed = private_allowlist();
let request = request_with(&[("host", &format!("[::ffff:192.168.1.5]:{PORT}"))]);
assert!(check_request_origin(&request, allowed.as_ref()).is_none());
}
#[test]
fn canonical_collapses_ipv4_mapped_loopback() {
// The `/api/open` peer gate relies on this: an IPv4 loopback client
// reaching a dual-stack (`::`) bind arrives as `::ffff:127.0.0.1`, which
// is only recognized as loopback after canonicalization.
let mapped: IpAddr = "::ffff:127.0.0.1".parse().unwrap();
assert!(!mapped.is_loopback());
assert!(canonical(mapped).is_loopback());
}
#[test]
fn normalize_host_strips_brackets_and_zone_id() {
// Brackets and an IPv6 zone id must be removed so the result parses as an
// `IpAddr` (`Ipv6Addr::from_str` rejects zone ids).
assert_eq!(normalize_host("[::1]"), "::1");
assert_eq!(normalize_host("[fe80::1%eth0]"), "fe80::1");
assert_eq!(normalize_host("localhost"), "localhost");
assert!(normalize_host("[fe80::1%eth0]").parse::<IpAddr>().is_ok());
}
#[test]
fn allowed_hosts_extend_the_allowlist() {
// A hostname listed as an allowed host is accepted on a loopback bind,
// with the usual port check still applied; an unlisted host is not.
let allowed = allowlist_with_hosts(IpAddr::V4(Ipv4Addr::LOCALHOST), &["mypc.lan"]);
let ok = request_with(&[("host", &format!("mypc.lan:{PORT}"))]);
assert!(check_request_origin(&ok, allowed.as_ref()).is_none());
let wrong_port = request_with(&[("host", "mypc.lan:1234")]);
assert!(check_request_origin(&wrong_port, allowed.as_ref()).is_some());
let other = request_with(&[("host", &format!("other.lan:{PORT}"))]);
assert!(check_request_origin(&other, allowed.as_ref()).is_some());
}
#[test]
fn allowed_hosts_apply_to_origin() {
let allowed = allowlist_with_hosts(IpAddr::V4(Ipv4Addr::LOCALHOST), &["mypc.lan"]);
let ok = request_with(&[
("host", &format!("mypc.lan:{PORT}")),
("origin", &format!("http://mypc.lan:{PORT}")),
]);
assert!(check_request_origin(&ok, allowed.as_ref()).is_none());
let foreign_origin = request_with(&[
("host", &format!("mypc.lan:{PORT}")),
("origin", &format!("http://evil.com:{PORT}")),
]);
assert!(check_request_origin(&foreign_origin, allowed.as_ref()).is_some());
}
#[test]
fn allowed_hosts_enable_enforcement_on_exposed_bind() {
// Binding to 0.0.0.0 normally disables validation, but listing a host
// turns it back on: localhost and the listed host are accepted while
// everything else is rejected.
let allowed = allowlist_with_hosts(IpAddr::V4(Ipv4Addr::UNSPECIFIED), &["mypc.lan"]);
assert!(allowed.is_some());
for host in [format!("mypc.lan:{PORT}"), format!("localhost:{PORT}")] {
let request = request_with(&[("host", &host)]);
assert!(
check_request_origin(&request, allowed.as_ref()).is_none(),
"host {host} should be allowed"
);
}
let evil = request_with(&[("host", "evil.com")]);
assert!(check_request_origin(&evil, allowed.as_ref()).is_some());
}
#[test]
fn allowed_hosts_on_public_bind_accept_the_bind_ip() {
// A specific public bind with an allowed host accepts localhost, the
// listed host, and the bind IP itself, but nothing else. (203.0.113.0/24
// is the TEST-NET-3 documentation range, treated here as a public IP.)
let bind = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 5));
let allowed = allowlist_with_hosts(bind, &["mypc.lan"]);
for host in [
format!("203.0.113.5:{PORT}"),
format!("mypc.lan:{PORT}"),
format!("localhost:{PORT}"),
] {
let request = request_with(&[("host", &host)]);
assert!(
check_request_origin(&request, allowed.as_ref()).is_none(),
"host {host} should be allowed"
);
}
let evil = request_with(&[("host", "evil.com")]);
assert!(check_request_origin(&evil, allowed.as_ref()).is_some());
}
#[test]
fn disabled_allowlist_accepts_foreign_host() {
for bind in [
IpAddr::V4(Ipv4Addr::UNSPECIFIED),
IpAddr::V4(Ipv4Addr::new(8, 8, 8, 8)),
] {
let allowed = allowed_hosts(bind, PORT, &[]);
let request = request_with(&[("host", "evil.com")]);
assert!(
check_request_origin(&request, allowed.as_ref()).is_none(),
"disabled allowlist for bind {bind} should accept any host"
);
}
}
}

View File

@@ -6,7 +6,7 @@
use std::{borrow::Cow, sync::Arc, time::Duration}; use std::{borrow::Cow, sync::Arc, time::Duration};
use hyper::{header, Body, Method, Request, Response, StatusCode}; use hyper::{Body, Method, Request, Response, StatusCode};
use rbx_dom_weak::types::{Ref, Variant}; use rbx_dom_weak::types::{Ref, Variant};
use ritz::{html, Fragment, HtmlContent, HtmlSelfClosingTag}; use ritz::{html, Fragment, HtmlContent, HtmlSelfClosingTag};
@@ -16,7 +16,7 @@ use crate::{
web::{ web::{
assets, assets,
interface::{ErrorResponse, SERVER_VERSION}, interface::{ErrorResponse, SERVER_VERSION},
util::json, util::{json, response},
}, },
}; };
@@ -45,17 +45,11 @@ impl UiService {
} }
fn handle_logo(&self) -> Response<Body> { fn handle_logo(&self) -> Response<Body> {
Response::builder() response(StatusCode::OK, "image/png", assets::logo())
.header(header::CONTENT_TYPE, "image/png")
.body(Body::from(assets::logo()))
.unwrap()
} }
fn handle_icon(&self) -> Response<Body> { fn handle_icon(&self) -> Response<Body> {
Response::builder() response(StatusCode::OK, "image/png", assets::icon())
.header(header::CONTENT_TYPE, "image/png")
.body(Body::from(assets::icon()))
.unwrap()
} }
fn handle_home(&self) -> Response<Body> { fn handle_home(&self) -> Response<Body> {
@@ -66,10 +60,11 @@ impl UiService {
</div> </div>
}); });
Response::builder() response(
.header(header::CONTENT_TYPE, "text/html") StatusCode::OK,
.body(Body::from(format!("<!DOCTYPE html>{}", page))) "text/html",
.unwrap() format!("<!DOCTYPE html>{}", page),
)
} }
fn handle_show_instances(&self) -> Response<Body> { fn handle_show_instances(&self) -> Response<Body> {
@@ -80,10 +75,11 @@ impl UiService {
{ Self::instance(&tree, root_id) } { Self::instance(&tree, root_id) }
}); });
Response::builder() response(
.header(header::CONTENT_TYPE, "text/html") StatusCode::OK,
.body(Body::from(format!("<!DOCTYPE html>{}", page))) "text/html",
.unwrap() format!("<!DOCTYPE html>{}", page),
)
} }
fn instance(tree: &RojoTree, id: Ref) -> HtmlContent<'_> { fn instance(tree: &RojoTree, id: Ref) -> HtmlContent<'_> {

View File

@@ -1,25 +1,78 @@
use hyper::{header::CONTENT_TYPE, Body, Response, StatusCode}; use hyper::{header::CONTENT_TYPE, Body, Response, StatusCode};
use serde::Serialize; use serde::{Deserialize, Serialize};
pub fn json_ok<T: Serialize>(value: T) -> Response<Body> { /// Builds an HTTP response, falling back to an empty `500` response (rather than
json(value, StatusCode::OK) /// panicking) if the response could not be constructed. With constant headers
/// and a valid status code this never actually fails, but routing every
/// response through here means a malformed response can never crash the server.
pub fn response(
code: StatusCode,
content_type: &'static str,
body: impl Into<Body>,
) -> Response<Body> {
Response::builder()
.status(code)
.header(CONTENT_TYPE, content_type)
.body(body.into())
.unwrap_or_else(|err| {
log::error!("Failed to build HTTP response: {}", err);
let mut fallback = Response::new(Body::empty());
*fallback.status_mut() = StatusCode::INTERNAL_SERVER_ERROR;
fallback
})
}
pub fn msgpack_ok<T: Serialize>(value: T) -> Response<Body> {
msgpack(value, StatusCode::OK)
}
pub fn msgpack<T: Serialize>(value: T, code: StatusCode) -> Response<Body> {
let mut serialized = Vec::new();
let mut serializer = rmp_serde::Serializer::new(&mut serialized)
.with_human_readable()
.with_struct_map();
if let Err(err) = value.serialize(&mut serializer) {
return response(
StatusCode::INTERNAL_SERVER_ERROR,
"text/plain",
err.to_string(),
);
};
response(code, "application/msgpack", serialized)
}
pub fn serialize_msgpack<T: Serialize>(value: T) -> anyhow::Result<Vec<u8>> {
let mut serialized = Vec::new();
let mut serializer = rmp_serde::Serializer::new(&mut serialized)
.with_human_readable()
.with_struct_map();
value.serialize(&mut serializer)?;
Ok(serialized)
}
pub fn deserialize_msgpack<'a, T: Deserialize<'a>>(
input: &'a [u8],
) -> Result<T, rmp_serde::decode::Error> {
let mut deserializer = rmp_serde::Deserializer::new(input).with_human_readable();
T::deserialize(&mut deserializer)
} }
pub fn json<T: Serialize>(value: T, code: StatusCode) -> Response<Body> { pub fn json<T: Serialize>(value: T, code: StatusCode) -> Response<Body> {
let serialized = match serde_json::to_string(&value) { let serialized = match serde_json::to_string(&value) {
Ok(v) => v, Ok(v) => v,
Err(err) => { Err(err) => {
return Response::builder() return response(
.status(StatusCode::INTERNAL_SERVER_ERROR) StatusCode::INTERNAL_SERVER_ERROR,
.header(CONTENT_TYPE, "text/plain") "text/plain",
.body(Body::from(err.to_string())) err.to_string(),
.unwrap(); );
} }
}; };
Response::builder() response(code, "application/json", serialized)
.status(code)
.header(CONTENT_TYPE, "application/json")
.body(Body::from(serialized))
.unwrap()
} }

View File

@@ -0,0 +1,14 @@
{
"name": "default",
"tree": {
"$className": "DataModel",
"ReplicatedStorage": {
"Project": {
"$path": "project/src",
"Module": {
"$path": "module"
}
}
}
}
}

View File

@@ -0,0 +1 @@
return nil

View File

@@ -0,0 +1,14 @@
{
"name": "default",
"tree": {
"$className": "DataModel",
"ReplicatedStorage": {
"Project": {
"$path": "src/",
"Module": {
"$path": "../module"
}
}
}
}
}

View File

@@ -0,0 +1 @@
return nil

View File

@@ -10,6 +10,7 @@ use std::{
use hyper_tungstenite::tungstenite::{connect, Message}; use hyper_tungstenite::tungstenite::{connect, Message};
use rbx_dom_weak::types::Ref; use rbx_dom_weak::types::Ref;
use serde::{Deserialize, Serialize};
use tempfile::{tempdir, TempDir}; use tempfile::{tempdir, TempDir};
use librojo::{ use librojo::{
@@ -125,6 +126,10 @@ impl TestServeSession {
&self.project_path &self.project_path
} }
pub fn port(&self) -> usize {
self.port
}
/// Waits for the `rojo serve` server to come online with expontential /// Waits for the `rojo serve` server to come online with expontential
/// backoff. /// backoff.
pub fn wait_to_come_online(&mut self) -> ServerInfoResponse { pub fn wait_to_come_online(&mut self) -> ServerInfoResponse {
@@ -161,22 +166,16 @@ impl TestServeSession {
pub fn get_api_rojo(&self) -> Result<ServerInfoResponse, reqwest::Error> { pub fn get_api_rojo(&self) -> Result<ServerInfoResponse, reqwest::Error> {
let url = format!("http://localhost:{}/api/rojo", self.port); let url = format!("http://localhost:{}/api/rojo", self.port);
let body = reqwest::blocking::get(url)?.text()?; let body = reqwest::blocking::get(url)?.bytes()?;
let value = jsonc_parser::parse_to_serde_value(&body, &Default::default()) Ok(deserialize_msgpack(&body).expect("Server returned malformed response"))
.expect("Failed to parse JSON")
.expect("No JSON value");
Ok(serde_json::from_value(value).expect("Server returned malformed response"))
} }
pub fn get_api_read(&self, id: Ref) -> Result<ReadResponse<'_>, reqwest::Error> { pub fn get_api_read(&self, id: Ref) -> Result<ReadResponse<'_>, reqwest::Error> {
let url = format!("http://localhost:{}/api/read/{}", self.port, id); let url = format!("http://localhost:{}/api/read/{}", self.port, id);
let body = reqwest::blocking::get(url)?.text()?; let body = reqwest::blocking::get(url)?.bytes()?;
let value = jsonc_parser::parse_to_serde_value(&body, &Default::default()) Ok(deserialize_msgpack(&body).expect("Server returned malformed response"))
.expect("Failed to parse JSON")
.expect("No JSON value");
Ok(serde_json::from_value(value).expect("Server returned malformed response"))
} }
pub fn get_api_socket_packet( pub fn get_api_socket_packet(
@@ -198,8 +197,8 @@ impl TestServeSession {
} }
match socket.read() { match socket.read() {
Ok(Message::Text(text)) => { Ok(Message::Binary(binary)) => {
let packet: SocketPacket = serde_json::from_str(&text)?; let packet: SocketPacket = deserialize_msgpack(&binary)?;
if packet.packet_type != packet_type { if packet.packet_type != packet_type {
continue; continue;
} }
@@ -212,7 +211,7 @@ impl TestServeSession {
return Err("WebSocket closed before receiving messages".into()); return Err("WebSocket closed before receiving messages".into());
} }
Ok(_) => { Ok(_) => {
// Ignore other message types (ping, pong, binary) // Ignore other message types (ping, pong, text)
continue; continue;
} }
Err(hyper_tungstenite::tungstenite::Error::Io(e)) Err(hyper_tungstenite::tungstenite::Error::Io(e))
@@ -229,20 +228,73 @@ impl TestServeSession {
} }
} }
pub fn get_api_serialize( pub fn post_api_serialize(
&self, &self,
ids: &[Ref], ids: &[Ref],
session_id: SessionId, session_id: SessionId,
) -> Result<SerializeResponse, reqwest::Error> { ) -> Result<reqwest::blocking::Response, reqwest::Error> {
let client = reqwest::blocking::Client::new(); let client = reqwest::blocking::Client::new();
let url = format!("http://localhost:{}/api/serialize", self.port); let url = format!("http://localhost:{}/api/serialize", self.port);
let body = serde_json::to_string(&SerializeRequest { let body = serialize_msgpack(&SerializeRequest {
session_id, session_id,
ids: ids.to_vec(), ids: ids.to_vec(),
}); })
.unwrap();
client.post(url).body((body).unwrap()).send()?.json() client.post(url).body(body).send()
} }
/// Sends a GET to `/api/rojo` with the given extra request headers and
/// returns the full response. Used to exercise the Host/Origin allowlist that
/// guards against DNS rebinding, including asserting that a rejection reveals
/// nothing about the server.
pub fn api_rojo_response_with_headers(
&self,
headers: &[(&str, &str)],
) -> reqwest::blocking::Response {
let client = reqwest::blocking::Client::new();
let url = format!("http://localhost:{}/api/rojo", self.port);
let mut request = client.get(url);
for (name, value) in headers {
request = request.header(*name, *value);
}
request.send().expect("Failed to send request")
}
/// Sends a POST to `/api/open/<id>` and returns the response status code.
/// Used to verify that the local-only gate on `/api/open` admits loopback
/// peers (the test harness always connects over loopback).
pub fn api_open_status(&self, id: &str) -> reqwest::StatusCode {
let client = reqwest::blocking::Client::new();
let url = format!("http://localhost:{}/api/open/{}", self.port, id);
client
.post(url)
.send()
.expect("Failed to send request")
.status()
}
}
fn serialize_msgpack<T: Serialize>(value: T) -> Result<Vec<u8>, rmp_serde::encode::Error> {
let mut serialized = Vec::new();
let mut serializer = rmp_serde::Serializer::new(&mut serialized)
.with_human_readable()
.with_struct_map();
value.serialize(&mut serializer)?;
Ok(serialized)
}
pub fn deserialize_msgpack<'a, T: Deserialize<'a>>(
input: &'a [u8],
) -> Result<T, rmp_serde::decode::Error> {
let mut deserializer = rmp_serde::Deserializer::new(input).with_human_readable();
T::deserialize(&mut deserializer)
} }
/// Probably-okay way to generate random enough port numbers for running the /// Probably-okay way to generate random enough port numbers for running the
@@ -262,11 +314,7 @@ fn get_port_number() -> usize {
/// Since the provided structure intentionally includes unredacted referents, /// Since the provided structure intentionally includes unredacted referents,
/// some post-processing is done to ensure they don't show up in the model. /// some post-processing is done to ensure they don't show up in the model.
pub fn serialize_to_xml_model(response: &SerializeResponse, redactions: &RedactionMap) -> String { pub fn serialize_to_xml_model(response: &SerializeResponse, redactions: &RedactionMap) -> String {
let model_content = data_encoding::BASE64 let mut dom = rbx_binary::from_reader(response.model_contents.as_slice()).unwrap();
.decode(response.model_contents.model().as_bytes())
.unwrap();
let mut dom = rbx_binary::from_reader(model_content.as_slice()).unwrap();
// This makes me realize that maybe we need a `descendants_mut` iter. // This makes me realize that maybe we need a `descendants_mut` iter.
let ref_list: Vec<Ref> = dom.descendants().map(|inst| inst.referent()).collect(); let ref_list: Vec<Ref> = dom.descendants().map(|inst| inst.referent()).collect();
for referent in ref_list { for referent in ref_list {

View File

@@ -65,6 +65,7 @@ gen_build_tests! {
no_name_default_project, no_name_default_project,
no_name_project, no_name_project,
no_name_top_level_project, no_name_top_level_project,
plugin_init,
} }
fn run_build_test(test_name: &str) { fn run_build_test(test_name: &str) {

View File

@@ -1,14 +1,84 @@
use std::fs; use std::fs;
use insta::{assert_snapshot, assert_yaml_snapshot, with_settings}; use insta::{assert_snapshot, assert_yaml_snapshot, with_settings};
use rbx_dom_weak::types::Ref;
use reqwest::StatusCode;
use tempfile::tempdir; use tempfile::tempdir;
use crate::rojo_test::{ use crate::rojo_test::{
internable::InternAndRedact, internable::InternAndRedact,
serve_util::{run_serve_test, serialize_to_xml_model}, serve_util::{deserialize_msgpack, run_serve_test, serialize_to_xml_model},
}; };
use librojo::web_api::SocketPacketType; use librojo::web_api::{SerializeResponse, SocketPacketType};
#[test]
fn rejects_dns_rebinding_requests() {
run_serve_test("empty", |session, _redactions| {
let port = session.port();
let local_host = format!("localhost:{port}");
// A request carrying a local Host header is served normally.
assert_eq!(
session
.api_rojo_response_with_headers(&[("host", &local_host)])
.status(),
reqwest::StatusCode::OK,
);
// A request whose Host is a foreign hostname, as a DNS-rebound page
// would send, is rejected with a generic 404 that reveals nothing about
// the server.
assert_rejected(session.api_rojo_response_with_headers(&[("host", "evil.com")]));
// Even with a local Host, a present-but-foreign Origin is rejected.
let foreign_origin = format!("http://evil.com:{port}");
assert_rejected(
session.api_rojo_response_with_headers(&[
("host", &local_host),
("origin", &foreign_origin),
]),
);
});
}
/// Asserts that a Host/Origin rejection is a generic 404 whose body and
/// content-type do not identify the server as Rojo.
fn assert_rejected(response: reqwest::blocking::Response) {
assert_eq!(response.status(), reqwest::StatusCode::NOT_FOUND);
let content_type = response
.headers()
.get(reqwest::header::CONTENT_TYPE)
.and_then(|value| value.to_str().ok())
.unwrap_or_default()
.to_owned();
assert!(
!content_type.contains("msgpack"),
"rejection should not use the msgpack API content-type, got {content_type:?}",
);
let body = response.text().expect("Failed to read response body");
let body_lower = body.to_lowercase();
assert!(
!body_lower.contains("rojo") && !body_lower.contains("rebinding"),
"rejection body should not identify the server, got {body:?}",
);
}
#[test]
fn allows_api_open_from_loopback_peer() {
run_serve_test("empty", |session, _redactions| {
// The harness always connects over loopback, so the local-only gate on
// /api/open must let the request through. A bogus instance id then fails
// id parsing with 400, which confirms we got past the gate rather than
// being rejected with 403.
assert_eq!(
session.api_open_status("not-a-real-ref"),
reqwest::StatusCode::BAD_REQUEST,
);
});
}
#[test] #[test]
fn empty() { fn empty() {
@@ -645,9 +715,13 @@ fn meshpart_with_id() {
.find(|(_, inst)| inst.class_name == "ObjectValue") .find(|(_, inst)| inst.class_name == "ObjectValue")
.unwrap(); .unwrap();
let serialize_response = session let body = session
.get_api_serialize(&[*meshpart, *objectvalue], info.session_id) .post_api_serialize(&[*meshpart, *objectvalue], info.session_id)
.unwrap()
.bytes()
.unwrap(); .unwrap();
let serialize_response: SerializeResponse =
deserialize_msgpack(&body).expect("Server returned malformed response");
// We don't assert a snapshot on the SerializeResponse because the model includes the // We don't assert a snapshot on the SerializeResponse because the model includes the
// Refs from the DOM as names, which means it will obviously be different every time // Refs from the DOM as names, which means it will obviously be different every time
@@ -659,6 +733,20 @@ fn meshpart_with_id() {
}); });
} }
#[test]
fn serialize_missing_id() {
run_serve_test("empty", |session, _| {
let info = session.get_api_rojo().unwrap();
let missing_id = Ref::new();
let response = session
.post_api_serialize(&[missing_id], info.session_id)
.unwrap();
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
});
}
#[test] #[test]
fn forced_parent() { fn forced_parent() {
run_serve_test("forced_parent", |session, mut redactions| { run_serve_test("forced_parent", |session, mut redactions| {
@@ -673,9 +761,13 @@ fn forced_parent() {
read_response.intern_and_redact(&mut redactions, root_id) read_response.intern_and_redact(&mut redactions, root_id)
); );
let serialize_response = session let body = session
.get_api_serialize(&[root_id], info.session_id) .post_api_serialize(&[root_id], info.session_id)
.unwrap()
.bytes()
.unwrap(); .unwrap();
let serialize_response: SerializeResponse =
deserialize_msgpack(&body).expect("Server returned malformed response");
assert_eq!(serialize_response.session_id, info.session_id); assert_eq!(serialize_response.session_id, info.session_id);

View File

@@ -86,4 +86,7 @@ syncback_tests! {
sync_rules => ["src/module.modulescript", "src/text.text"], sync_rules => ["src/module.modulescript", "src/text.text"],
// Ensures that the `syncUnscriptable` setting works // Ensures that the `syncUnscriptable` setting works
unscriptable_properties => ["default.project.json"], unscriptable_properties => ["default.project.json"],
// Ensures that syncback correctly removes default values from projects rather
// than leaving an incorrect value.
project_default_properties_remove => ["default.project.json"],
} }