Compare commits

...

13 Commits

Author SHA1 Message Date
Lucien Greathouse
bf9be6ccae Fix reconciler with init files, v0.2.2 2017-12-01 15:18:36 -08:00
Lucien Greathouse
974ebc33c2 Major documentation facelift, should be usable now 2017-12-01 14:07:06 -08:00
Lucien Greathouse
4b03a79cfe Change config to work with plugin version v0.2.1 2017-12-01 02:49:49 -08:00
Lucien Greathouse
43cc350b7a 0.2.1 2017-12-01 02:48:43 -08:00
Lucien Greathouse
5685619c3a Switch to using the latest Rojo release to sync itself 2017-12-01 02:40:08 -08:00
Lucien Greathouse
f3483ee2e0 0.2.0 2017-12-01 02:02:39 -08:00
Lucien Greathouse
60a9135452 Robust init.lua support 2017-12-01 01:55:34 -08:00
Lucien Greathouse
c3d6dc0e2c First past at implementing init.lua support 2017-12-01 01:28:23 -08:00
Lucien Greathouse
2681972976 Much more robust reconciliation implementation 2017-12-01 00:53:41 -08:00
Lucien Greathouse
5e64773784 Improve plugin accuracy 2017-12-01 00:18:11 -08:00
Lucien Greathouse
c7171ef513 Add Rojo config for testing 2017-12-01 00:17:45 -08:00
Lucien Greathouse
63b21b90ff Ripple verbosity flags through the server 2017-12-01 00:17:29 -08:00
Lucien Greathouse
7f3aaf4680 Fix Cargo metadata 2017-11-29 17:41:30 -08:00
15 changed files with 458 additions and 152 deletions

View File

@@ -3,5 +3,17 @@
## Current Master
* *No changes*
## 0.2.2
* Plugin only release
* Fixed broken reconciliation behavior with `init` files
## 0.2.1
* Plugin only release
* Changes default port to 8000
## 0.2.0
* Support for `init.lua` like rbxfs and rbxpacker
* More robust syncing with a new reconciler
## 0.1.0
* Initial release, functionally very similar to [rbxfs](https://github.com/LPGhatguy/rbxfs)

2
Cargo.lock generated
View File

@@ -1,6 +1,6 @@
[root]
name = "rojo"
version = "0.1.0"
version = "0.2.0"
dependencies = [
"clap 2.28.0 (registry+https://github.com/rust-lang/crates.io-index)",
"notify 4.0.3 (registry+https://github.com/rust-lang/crates.io-index)",

View File

@@ -1,8 +1,10 @@
[package]
name = "rojo"
version = "0.1.0"
version = "0.2.0"
authors = ["Lucien Greathouse <me@lpghatguy.com>"]
description = "A tool to create robust Roblox projects"
license = "MIT"
repository = "https://github.com/LPGhatguy/rojo"
[[bin]]
name = "rojo"

123
README.md
View File

@@ -3,53 +3,37 @@
<a href="https://travis-ci.org/LPGhatguy/Rojo">
<img src="https://api.travis-ci.org/LPGhatguy/Rojo.svg?branch=master" alt="Travis-CI Build Status" />
</a>
<a href="#">
<img src="https://img.shields.io/badge/docs-soon-red.svg" alt="Documentation" />
</a>
</div>
<div>&nbsp;</div>
**EARLY DEVELOPMENT, USE WITH CARE**
Rojo is a flexible multi-tool designed for creating robust Roblox projects. It's in early development, but is still useful for many projects.
Rojo is a flexible multi-tool designed for creating robust Roblox projects.
It's designed for power users who want to use the **best tools available** for building games, libraries, and plugins.
It's designed for power users who want to use the best tools available for building games, libraries, and plugins.
## Features
It has a number of desirable features *right now*:
Rojo has a number of desirable features *right now*:
* Work from the filesystem, in your favorite editor
* Work on scripts from the filesystem, in your favorite editor
* Version your place, library, or plugin using Git or another VCS
Soon, Rojo will be able to:
* Sync Roblox objects (including models) bi-directionally between the filesystem and Roblox Studio
* Create installation scripts for libraries to be used in standalone places
* Similar to [rbxpacker](https://github.com/LPGhatguy/rbxpacker), another one of my projects
* Add strongly-versioned dependencies to your project
## Installation
Rojo has two components:
* The binary, written in Rust
* The command line tool, written in Rust
* The [Roblox Studio plugin](https://www.roblox.com/library/1211549683/Rojo-v0-0-0), written in Lua
To install the binary, there are two options:
* Cargo, which requires you to have Rust installed
* Pre-built binaries from the [the GitHub releases page](https://github.com/LPGhatguy/rojo/releases)
### Cargo (Recommended)
Make sure you have [Rust 1.21 or newer](https://www.rust-lang.org/) installed.
Install Rojo using:
```sh
cargo install rojo
# Installed!
rojo help
```
### Pre-Built (Windows only)
Download the latest binary from [the GitHub releases page](https://github.com/LPGhatguy/rojo/releases). Put it somewhere you can access it from a terminal!
To install the command line tool, there are two options:
* Cargo, if you have Rust installed
* Use `cargo install rojo` -- Rojo will be available with the `rojo` command
* Download a pre-built Windows binary from [the GitHub releases page](https://github.com/LPGhatguy/rojo/releases)
## Usage
For more help, use `rojo help`.
@@ -66,8 +50,91 @@ rojo init
Rojo will create an empty project in the directory.
The default project looks like this:
```json
{
"name": "my-new-project",
"servePort": 8000,
"partitions": {}
}
```
### Start Dev Server
To create a server that allows the Rojo Dev Plugin to access your project, use:
```sh
rojo serve
```
The tool will tell you whether it found an existing project. You should then be able to connect and use the project from within Roblox Studio!
### Migrating an Existing Roblox Project
Coming soon!
**Coming soon!**
### Syncing into Roblox
In order to sync code into Roblox, you'll need to add one or more "partitions" to your configuration. A partition tells Rojo how to map directories to Roblox objects.
Each entry in the partitions table has a unique name, a filesystem path, and the full name of the Roblox object to sync into.
For example, if you want to map your `src` directory to an object named `My Cool Game` in `ReplicatedStorage`, you could use this configuration:
```json
{
"name": "rojo",
"servePort": 8000,
"partitions": {
"game": {
"path": "src",
"target": "ReplicatedStorage.My Cool Game"
}
}
}
```
The `path` parameter is relative to the project file.
The `target` starts at `game` and crawls down the tree. If any objects don't exist along the way, they'll be created as `Folder` instances.
Run `rojo serve` in the directory containing this project, then press the "Sync In" or "Toggle Polling" buttons in the Roblox Studio plugin to move code into your game.
### Sync Details
The structure of files and folders on the filesystem are preserved when syncing into game.
Creation of Roblox instances follows a simple set of rules. The first rule that matches the file name is chosen:
| File Name | Instance Type | Notes |
| -------------- | -------------- | ----------------------------------------- |
| `*.server.lua` | `Script` | `Source` will contain the file's contents |
| `*.client.lua` | `LocalScript` | `Source` will contain the file's contents |
| `*.lua` | `ModuleScript` | `Source` will contain the file's contents |
| `*` | `StringValue` | `Value` will contain the file's contents |
Any folders on the filesystem will turn into `Folder` objects unless they contain a file named `init` with any extension. Following the convention of Lua, those objects will instead be whatever the `init` file would turn into.
For example, this file tree:
* my-game
* init.lua
* foo.lua
Will turn into this tree in Roblox:
* `my-game` (`ModuleScript` with source from `my-game/init.lua`)
* `foo` (`ModuleScript` with source from `my-game/foo.lua`)
## Inspiration
There are lots of other tools that sync scripts into Roblox, or otherwise work to improve the development flow outside of Roblox Studio.
Here are a few, if you're looking for alternatives or supplements to Rojo:
* [rbxmk by Anaminus](https://github.com/anaminus/rbxmk)
* [RbxRefresh by Osyris](https://github.com/osyrisrblx/RbxRefresh)
* [RbxSync by evaera](https://github.com/evaera/RbxSync)
* [CodeSync](https://github.com/MemoryPenguin/CodeSync) and [rbx-exteditor](https://github.com/MemoryPenguin/rbx-exteditor) by [MemoryPenguin](https://github.com/MemoryPenguin)
I also have a couple tools that Rojo intends to replace:
* [rbxfs](https://github.com/LPGhatguy/rbxfs), which has been deprecated by Rojo
* [rbxpacker](https://github.com/LPGhatguy/rbxpacker), which is still useful
## License
Rojo is available under the terms of the MIT license. See [LICENSE.md](LICENSE.md) for details.

View File

@@ -1,4 +0,0 @@
{
"rootDirectory": "src",
"rootObject": "ReplicatedStorage.Rojo"
}

View File

@@ -1,3 +1,4 @@
return {
pollingRate = 0.3,
version = "0.2.2",
}

View File

@@ -1,9 +1,17 @@
local HttpService = game:GetService("HttpService")
local HTTP_DEBUG = false
local Promise = require(script.Parent.Promise)
local HttpError = require(script.Parent.HttpError)
local HttpResponse = require(script.Parent.HttpResponse)
local function dprint(...)
if HTTP_DEBUG then
print(...)
end
end
local Http = {}
Http.__index = Http
@@ -20,6 +28,7 @@ function Http.new(baseUrl)
end
function Http:get(endpoint)
dprint("\nGET", endpoint)
return Promise.new(function(resolve, reject)
spawn(function()
local ok, result = pcall(function()
@@ -27,6 +36,7 @@ function Http:get(endpoint)
end)
if ok then
dprint("\t", result, "\n")
resolve(HttpResponse.new(result))
else
reject(HttpError.fromErrorString(result))
@@ -36,6 +46,8 @@ function Http:get(endpoint)
end
function Http:post(endpoint, body)
dprint("\nPOST", endpoint)
dprint(body)
return Promise.new(function(resolve, reject)
spawn(function()
local ok, result = pcall(function()
@@ -43,6 +55,7 @@ function Http:post(endpoint, body)
end)
if ok then
dprint("\t", result, "\n")
resolve(HttpResponse.new(result))
else
reject(HttpError.fromErrorString(result))

View File

@@ -3,26 +3,36 @@ if not plugin then
end
local Plugin = require(script.Parent.Plugin)
local Config = require(script.Parent.Config)
local function main()
local pluginInstance = Plugin.new()
local toolbar = plugin:CreateToolbar("Rojo Plugin 0.1.0")
local toolbar = plugin:CreateToolbar("Rojo Plugin v" .. Config.version)
toolbar:CreateButton("Test Connection", "Connect to Rojo Server", "")
.Click:Connect(function()
pluginInstance:connect()
:catch(function(err)
warn(err)
end)
end)
toolbar:CreateButton("Sync In", "Sync into Roblox Studio", "")
.Click:Connect(function()
pluginInstance:syncIn()
:catch(function(err)
warn(err)
end)
end)
toolbar:CreateButton("Toggle Polling", "Poll server for changes", "")
.Click:Connect(function()
spawn(function()
pluginInstance:togglePolling()
:catch(function(err)
warn(err)
end)
end)
end)
end

View File

@@ -2,6 +2,7 @@ local Config = require(script.Parent.Config)
local Http = require(script.Parent.Http)
local Server = require(script.Parent.Server)
local Promise = require(script.Parent.Promise)
local Reconciler = require(script.Parent.Reconciler)
local function collectMatch(source, pattern)
local result = {}
@@ -13,91 +14,12 @@ local function collectMatch(source, pattern)
return result
end
local function fileToName(filename)
if filename:find("%.server%.lua$") then
return filename:match("^(.-)%.server%.lua$"), "Script"
elseif filename:find("%.client%.lua$") then
return filename:match("^(.-)%.client%.lua$"), "LocalScript"
elseif filename:find("%.lua") then
return filename:match("^(.-)%.lua$"), "ModuleScript"
else
return filename, "StringValue"
end
end
local function nameToInstance(filename, contents)
local name, className = fileToName(filename)
local instance = Instance.new(className)
instance.Name = name
if className:find("Script$") then
instance.Source = contents
else
instance.Value = contents
end
return instance
end
local function make(item, name)
if item.type == "dir" then
local instance = Instance.new("Folder")
instance.Name = name
for childName, child in pairs(item.children) do
make(child, childName).Parent = instance
end
return instance
elseif item.type == "file" then
return nameToInstance(name, item.contents)
else
error("not implemented")
end
end
local function write(parent, route, item)
local location = parent
for index = 1, #route - 1 do
local piece = route[index]
local newLocation = location:FindFirstChild(piece)
if not newLocation then
newLocation = Instance.new("Folder")
newLocation.Name = piece
newLocation.Parent = location
end
location = newLocation
end
local fileName = route[#route]
local name = fileToName(fileName)
local existing = location:FindFirstChild(name)
local new
if item then
new = make(item, fileName)
end
if existing then
existing:Destroy()
end
if new then
new.Parent = location
end
end
local Plugin = {}
Plugin.__index = Plugin
function Plugin.new()
local address = "localhost"
local port = 8081
local port = 8000
local remote = ("http://%s:%d"):format(address, port)
@@ -149,7 +71,7 @@ end
function Plugin:connect()
print("Testing connection...")
self:server()
return self:server()
:andThen(function(server)
return server:getInfo()
end)
@@ -163,8 +85,10 @@ end
function Plugin:togglePolling()
if self._polling then
self:stopPolling()
return Promise.resolve(nil)
else
self:startPolling()
return self:startPolling()
end
end
@@ -173,12 +97,30 @@ function Plugin:stopPolling()
return
end
print("Stopping polling...")
print("Stopped polling.")
self._polling = false
self._label.Enabled = false
end
function Plugin:_pull(server, project, routes)
local items = server:read(routes):await()
for index = 1, #routes do
local route = routes[index]
local partitionName = route[1]
local partition = project.partitions[partitionName]
local item = items[index]
local fullRoute = collectMatch(partition.target, "[^.]+")
for i = 2, #route do
table.insert(fullRoute, routes[index][i])
end
Reconciler.reconcileRoute(fullRoute, item)
end
end
function Plugin:startPolling()
if self._polling then
return
@@ -204,17 +146,7 @@ function Plugin:startPolling()
table.insert(routes, change.route)
end
local items = server:read(routes):await()
for index = 1, #routes do
local partitionName = routes[index][1]
local partition = project.partitions[partitionName]
local data = items[index]
local fullRoute = collectMatch(partition.target, "[^.]+")
write(game, fullRoute, data)
end
self:_pull(server, project, routes)
wait(Config.pollingRate)
end
@@ -231,23 +163,13 @@ function Plugin:syncIn()
:andThen(function(server)
local project = server:getInfo():await().project
local readRoutes = {}
local routes = {}
for name in pairs(project.partitions) do
table.insert(readRoutes, {name})
table.insert(routes, {name})
end
local items = server:read(readRoutes):await()
for index = 1, #readRoutes do
local partitionName = readRoutes[index][1]
local partition = project.partitions[partitionName]
local data = items[index]
local fullRoute = collectMatch(partition.target, "[^.]+")
write(game, fullRoute, data)
end
self:_pull(server, project, routes)
print("Sync successful!")
end)

View File

@@ -2,7 +2,7 @@
An implementation of Promises similar to Promise/A+.
]]
local PROMISE_DEBUG = true
local PROMISE_DEBUG = false
-- If promise debugging is on, use a version of pcall that warns on failure.
-- This is useful for finding errors that happen within Promise itself.
@@ -89,6 +89,9 @@ function Promise.new(callback)
-- Only valid if _status is set to something besides Started
_value = nil,
-- If an error occurs with no observers, this will be set.
_unhandledRejection = false,
-- Queues representing functions we should invoke when we update!
_queuedResolve = {},
_queuedReject = {},
@@ -157,6 +160,8 @@ end
The given callbacks are invoked depending on that result.
]]
function Promise:andThen(successHandler, failureHandler)
self._unhandledRejection = false
-- Create a new promise to follow this part of the chain
return Promise.new(function(resolve, reject)
-- Our default callbacks just pass values onto the next promise.
@@ -199,6 +204,8 @@ end
This matches the execution model of normal Roblox functions.
]]
function Promise:await()
self._unhandledRejection = false
if self._status == Promise.Status.Started then
local result
local bindable = Instance.new("BindableEvent")
@@ -279,11 +286,22 @@ function Promise:_reject(...)
-- synchronously. We'll wait one tick, and if there are still no
-- observers, then we should put a message in the console.
local message = ("Unhandled promise rejection:\n\n%s\n\n%s"):format(
tostring((...)),
self._source
)
warn(message)
self._unhandledRejection = true
local err = tostring((...))
spawn(function()
-- Someone observed the error, hooray!
if not self._unhandledRejection then
return
end
-- Build a reasonable message
local message = ("Unhandled promise rejection:\n\n%s\n\n%s"):format(
err,
self._source
)
warn(message)
end)
end
end

237
plugin/src/Reconciler.lua Normal file
View File

@@ -0,0 +1,237 @@
local Reconciler = {}
local function isInit(item, itemFileName)
if item and item.type == "dir" then
return
end
return not not itemFileName:find("^init%.")
end
local function findInit(item)
if item.type ~= "dir" then
return nil, nil
end
for childFileName, childItem in pairs(item.children) do
if isInit(childItem, childFileName) then
return childItem, childFileName
end
end
return nil, nil
end
local function itemToName(item, fileName)
if item and item.type == "dir" then
return fileName, "Folder"
elseif item and item.type == "file" or not item then
if fileName:find("%.server%.lua$") then
return fileName:match("^(.-)%.server%.lua$"), "Script"
elseif fileName:find("%.client%.lua$") then
return fileName:match("^(.-)%.client%.lua$"), "LocalScript"
elseif fileName:find("%.lua") then
return fileName:match("^(.-)%.lua$"), "ModuleScript"
else
return fileName, "StringValue"
end
else
error("unknown item type " .. tostring(item.type))
end
end
local function setValues(rbx, item, fileName)
local _, className = itemToName(item, fileName)
if className:find("Script") then
rbx.Source = item.contents
else
rbx.Value = item.contents
end
end
function Reconciler._reifyShallow(item, fileName)
if item.type == "dir" then
local initItem, initFileName = findInit(item)
if initItem then
local rbx = Reconciler._reify(initItem, initFileName)
rbx.Name = fileName
return rbx
else
local rbx = Instance.new("Folder")
rbx.Name = fileName
return rbx
end
elseif item.type == "file" then
local objectName, className = itemToName(item, fileName)
local rbx = Instance.new(className)
rbx.Name = objectName
setValues(rbx, item, fileName)
return rbx
else
error("unknown item type " .. tostring(item.type))
end
end
function Reconciler._reify(item, fileName, parent)
local rbx = Reconciler._reifyShallow(item, fileName)
if item.type == "dir" then
for childFileName, childItem in pairs(item.children) do
if not isInit(childItem, childFileName) then
local childRbx = Reconciler._reify(childItem, childFileName)
childRbx.Parent = rbx
end
end
end
rbx.Parent = parent
return rbx
end
function Reconciler.reconcile(rbx, item, fileName, parent)
-- Item was deleted!
if not item then
if isInit(item, fileName) then
if not parent then
return
end
-- Un-usurp parent!
local newParent = Instance.new("Folder")
newParent.Name = parent.Name
for _, child in ipairs(parent:GetChildren()) do
child.Parent = newParent
end
newParent.Parent = parent.Parent
parent:Destroy()
return
else
if rbx then
rbx:Destroy()
end
return
end
end
if item.type == "dir" then
-- Folder was created!
if not rbx then
return Reconciler._reify(item, fileName, parent)
end
local initItem, initFileName = findInit(item)
if initItem then
local _, initClassName = itemToName(initItem, initFileName)
if rbx.ClassName == initClassName then
setValues(rbx, initItem, initFileName)
else
rbx:Destroy()
return Reconciler._reify(item, fileName, parent)
end
else
if rbx.ClassName ~= "Folder" then
rbx:Destroy()
return Reconciler._reify(item, fileName, parent)
end
end
local visitedChildren = {}
for childFileName, childItem in pairs(item.children) do
if not isInit(childItem, childFileName) then
local childName = itemToName(childItem, childFileName)
visitedChildren[childName] = true
Reconciler.reconcile(rbx:FindFirstChild(childName), childItem, childFileName, rbx)
end
end
for _, childRbx in ipairs(rbx:GetChildren()) do
-- Child was deleted!
if not visitedChildren[childRbx.Name] then
childRbx:Destroy()
end
end
return rbx
elseif item.type == "file" then
if isInit(item, fileName) then
-- Usurp our container!
local _, className = itemToName(item, fileName)
if parent.ClassName == className then
rbx = parent
else
rbx = Reconciler._reify(item, fileName, parent.Parent)
rbx.Name = parent.Name
for _, child in ipairs(parent:GetChildren()) do
child.Parent = rbx
end
parent:Destroy()
end
setValues(rbx, item, fileName)
return rbx
else
if not rbx then
return Reconciler._reify(item, fileName, parent)
end
local _, className = itemToName(item, fileName)
if rbx.ClassName ~= className then
rbx:Destroy()
return Reconciler._reify(item, fileName, parent)
end
setValues(rbx, item, fileName)
return rbx
end
else
error("unknown item type " .. tostring(item.type))
end
end
function Reconciler.reconcileRoute(route, item)
local location = game
for i = 1, #route - 1 do
local piece = route[i]
local newLocation = location:FindFirstChild(piece)
if not newLocation then
newLocation = Instance.new("Folder")
newLocation.Name = piece
newLocation.Parent = location
end
location = newLocation
end
local fileName = route[#route]
local name = itemToName(item, fileName)
local rbx = location:FindFirstChild(name)
Reconciler.reconcile(rbx, item, fileName, location)
end
return Reconciler

10
rojo.json Normal file
View File

@@ -0,0 +1,10 @@
{
"name": "rojo",
"servePort": 8000,
"partitions": {
"plugin": {
"path": "plugin/src",
"target": "ReplicatedStorage.Rojo Plugin"
}
}
}

View File

@@ -128,7 +128,7 @@ fn main() {
}
let vfs = {
let mut vfs = Vfs::new();
let mut vfs = Vfs::new(config.clone());
for (name, project_partition) in &project.partitions {
let path = {
@@ -158,9 +158,10 @@ fn main() {
{
let vfs = vfs.clone();
let config = config.clone();
thread::spawn(move || {
VfsWatcher::new(vfs).start();
VfsWatcher::new(config, vfs).start();
});
}

View File

@@ -5,6 +5,8 @@ use std::io::Read;
use std::path::{Path, PathBuf};
use std::time::Instant;
use core::Config;
/// Represents a virtual layer over multiple parts of the filesystem.
///
/// Paths in this system are represented as slices of strings, and are always
@@ -21,6 +23,8 @@ pub struct Vfs {
/// A chronologically-sorted list of routes that changed since the Vfs was
/// created, along with a timestamp denoting when.
pub change_history: Vec<VfsChange>,
config: Config,
}
#[derive(Debug, Serialize, Deserialize)]
@@ -38,11 +42,12 @@ pub enum VfsItem {
}
impl Vfs {
pub fn new() -> Vfs {
pub fn new(config: Config) -> Vfs {
Vfs {
partitions: HashMap::new(),
start_time: Instant::now(),
change_history: Vec::new(),
config,
}
}
@@ -140,6 +145,10 @@ impl Vfs {
}
pub fn add_change(&mut self, timestamp: f64, route: Vec<String>) {
if self.config.verbose {
println!("Added change {:?}", route);
}
self.change_history.push(VfsChange {
timestamp,
route,

View File

@@ -6,17 +6,20 @@ use notify::{DebouncedEvent, RecommendedWatcher, RecursiveMode, Watcher};
use vfs::Vfs;
use pathext::path_to_route;
use core::Config;
pub struct VfsWatcher {
vfs: Arc<Mutex<Vfs>>,
watchers: Vec<RecommendedWatcher>,
config: Config,
}
impl VfsWatcher {
pub fn new(vfs: Arc<Mutex<Vfs>>) -> VfsWatcher {
pub fn new(config: Config, vfs: Arc<Mutex<Vfs>>) -> VfsWatcher {
VfsWatcher {
vfs,
watchers: Vec::new(),
config,
}
}
@@ -40,6 +43,7 @@ impl VfsWatcher {
{
let vfs = self.vfs.clone();
let config = self.config.clone();
thread::spawn(move || {
loop {
@@ -47,6 +51,10 @@ impl VfsWatcher {
let mut vfs = vfs.lock().unwrap();
let current_time = vfs.current_time();
if config.verbose {
println!("FS event {:?}", event);
}
match event {
DebouncedEvent::Write(ref change_path) |
DebouncedEvent::Create(ref change_path) |