Dealing with and preventing mod incompatibility

[Thundercraft]

While Avorion does sort of support mods, it seems safe to say that it does not have true mod support yet - not like other games.

 

The biggest issue with mods right now, I think, is the potential for mod incompatibility. That is, more than one mod trying to change the same files. But there are other concerns, such as when Avorion gets an update and your modded files are not updated to work with the new version. Also, there's the issue of uninstalling a mod such as when it stops working or you no longer want to use it.

 

Some mods which I've seen that try to change the same files:

• Out of sector production, Carrier Commands, and Highlite Players all replace server.lua.
  
• Better Shipyard and Spawn Changer both replace shipyard.lua.

Perhaps the solution would be for more mod authors to do like guiohm with Allow Multi Delivery by using a file comparison tool to compare their modified files with vanilla files and creating .diff files for players to patch their own game files (after making backups)? In theory, this might also allow mods to keep up with version changes as Avorion gets updated - to a very limited extent, anyway.

[Laserzwei]

A semi-good inbetween solution would be if we had a folder wherein every lua file gets called once e.g. on serverstart. So we modders can set up our hooks(Callbacks) into the game and don't depend on modifying existing .lua files.

[Dahak]

Really, we probably need to mod most hardcoded lists in the scripts into registries. Probably not too hard. Actually probably just more time consuming than hard, given Lua's amazing table handling. Though we'll still need the above suggestion for a folder that gets called automatically. :/

[MMaster]

This is exactly why I've made the Mod Patch Generate & Apply batch tool (http://www.avorion.net/forum/index.php/topic,1304.0.html). It automatically generates diffs against vanilla game for each mod and then patches the game using those diffs instead of just replacing files. It works great so far.

 

I definitely support idea of distributing diffs instead of whole files. There needs to be some standard established soon otherwise it will become unmanageable. I've already exchanged some messages with bit that works on mod management tool. My suggestion was to take a look at tool like CKAN for Kerbal Space Program and maybe make a fork of it for Avorion.

[koonschi]

Okay, so, let's discuss this.

 

I think we can all agree that perfect mod compatibility is generally impossible. Imagine two mods doing the exact opposite of each other, and you've got a conflict, easy as that.

 

But we can make it so that conflicts appear much less often. I've spent quite some time to think on this and my proposal is this:

Since in lua, you can just redefine a function, it would be possible to read one script, and then read another script. Imagine 2 scripts that tinker with shipyard.lua. They both have to inject the update() function. What you could do is, in the second script, save the update function and write your own that calls the old one. Like this: 

local updateOld = update
function update(timeStep) 
    updateOld(timeStep)
    print("this is my custom output")
end 

 

The new update() function would replace the old one and call it. The neat thing is: You can repeat this as often as you want actually.

 

This would be the start. In order for this to work, I'd suggest a second folder which is left untouched by steam updates, so you don't have to reinstall all your mods every time a new update hits. Maybe even a separate folder for every mod that's downloaded when logging into a server, so you have your server mods but they won't pollute your custom games.

 

The game would load the vanilla script and then load all the modified ones it finds, afterwards. You could, for example, name them shipyard.lua.1 and shipyard.lua.2. Those lie around in the mods folder, and Avorion looks for them, sorts them lexicographically so load order is deterministic, and loads them as well in deterministic order after the vanilla one, into the same VM (or lua_State).

 

This could work for all scripts, and you wouldn't need diffs.

 

Any objections? Anything that wouldn't work with this system?

[MMaster]

Maybe I misunderstood, but there is one thing I don't like very much about it is the 'shipyard.lua.1' 'shipyard.lua.2' because that would mean that mods still need to make the file names unique between mods.

 

If I can suggest I would like to see each mod having its own folder with similar folder structure as game and the game automatically loading scripts just as you described but from mod folders. That way the mod can still call the file 'shipyard.lua'.

This way each mod will have its own directory that can contain even other content, like textures, sounds etc and it will be easy to manage.

I know that it may be a bit more work, but in my opinion it would be worth it.

 

Otherwise it sounds great, it's definitely better than what we have now :)

[Tarantel]

While Avorion does sort of support mods, it seems safe to say that it does not have true mod support yet - not like other games.

 

The biggest issue with mods right now, I think, is the potential for mod incompatibility. That is, more than one mod trying to change the same files. But there are other concerns, such as when Avorion gets an update and your modded files are not updated to work with the new version. Also, there's the issue of uninstalling a mod such as when it stops working or you no longer want to use it.

 

Some mods which I've seen that try to change the same files:

• Out of sector production, Carrier Commands, and Highlite Players all replace server.lua.
  
• Better Shipyard and Spawn Changer both replace shipyard.lua.

Perhaps the solution would be for more mod authors to do like guiohm with Allow Multi Delivery by using a file comparison tool to compare their modified files with vanilla files and creating .diff files for players to patch their own game files (after making backups)? In theory, this might also allow mods to keep up with version changes as Avorion gets updated - to a very limited extent, anyway.

 

I added support for Theoman02's Spawn Changer ^^ check my Post ^^

[Dahak]

Okay, so, let's discuss this.

 

I think we can all agree that perfect mod compatibility is generally impossible. Imagine two mods doing the exact opposite of each other, and you've got a conflict, easy as that.

 

But we can make it so that conflicts appear much less often. I've spent quite some time to think on this and my proposal is this:

Since in lua, you can just redefine a function, it would be possible to read one script, and then read another script. Imagine 2 scripts that tinker with shipyard.lua. They both have to inject the update() function. What you could do is, in the second script, save the update function and write your own that calls the old one. Like this: 

local updateOld = update
function update(timeStep) 
    updateOld(timeStep)
    print("this is my custom output")
end 

 

The new update() function would replace the old one and call it. The neat thing is: You can repeat this as often as you want actually.

 

This would be the start. In order for this to work, I'd suggest a second folder which is left untouched by steam updates, so you don't have to reinstall all your mods every time a new update hits. Maybe even a separate folder for every mod that's downloaded when logging into a server, so you have your server mods but they won't pollute your custom games.

 

The game would load the vanilla script and then load all the modified ones it finds, afterwards. You could, for example, name them shipyard.lua.1 and shipyard.lua.2. Those lie around in the mods folder, and Avorion looks for them, sorts them lexicographically so load order is deterministic, and loads them as well in deterministic order after the vanilla one, into the same VM (or lua_State).

 

This could work for all scripts, and you wouldn't need diffs.

 

Any objections? Anything that wouldn't work with this system?

 

I think I agree with MMaster - but like him, I'm not sure I'm understanding that well.

 

I would be perfectly happy with mod folders and a "mod.lua" file that's automatically loaded that tells the game how to interpret the mods.

 

 

[Nexus]

Okay, so, let's discuss this.

 

I think we can all agree that perfect mod compatibility is generally impossible. Imagine two mods doing the exact opposite of each other, and you've got a conflict, easy as that.

 

But we can make it so that conflicts appear much less often. I've spent quite some time to think on this and my proposal is this:

Since in lua, you can just redefine a function, it would be possible to read one script, and then read another script. Imagine 2 scripts that tinker with shipyard.lua. They both have to inject the update() function. What you could do is, in the second script, save the update function and write your own that calls the old one. Like this: 

local updateOld = update
function update(timeStep) 
    updateOld(timeStep)
    print("this is my custom output")
end 

 

The new update() function would replace the old one and call it. The neat thing is: You can repeat this as often as you want actually.

 

This would be the start. In order for this to work, I'd suggest a second folder which is left untouched by steam updates, so you don't have to reinstall all your mods every time a new update hits. Maybe even a separate folder for every mod that's downloaded when logging into a server, so you have your server mods but they won't pollute your custom games.

 

The game would load the vanilla script and then load all the modified ones it finds, afterwards. You could, for example, name them shipyard.lua.1 and shipyard.lua.2. Those lie around in the mods folder, and Avorion looks for them, sorts them lexicographically so load order is deterministic, and loads them as well in deterministic order after the vanilla one, into the same VM (or lua_State).

 

This could work for all scripts, and you wouldn't need diffs.

 

Any objections? Anything that wouldn't work with this system?

 

I like it, although if all this modding takes off. We're going to need some better handling if a script suddenly goes missing from a ship =P

Carrier Commands mod has proven binding things to player ships/players themselves is dangerous.

 

EDIT: So for carrier commands i would do something like this:

--Filename: server.lua.override < does it have to be a number?>
local onPlayerLogInOld = onPlayerLogIn
function onPlayerLogIn(playerIndex)
    onPlayerLogInOld(playerIndex)
    local player = Player(playerIndex)
    player.addScriptOnce("player/scriptloader.lua")
end

 

[koonschi]

I like it, although if all this modding takes off. We're going to need some better handling if a script suddenly goes missing from a ship =P

Carrier Commands mod has proven binding things to player ships/players themselves is dangerous.

What happened?

[Nexus]

I like it, although if all this modding takes off. We're going to need some better handling if a script suddenly goes missing from a ship =P

Carrier Commands mod has proven binding things to player ships/players themselves is dangerous.

What happened?

Well if someone tries to remove the mod the server errors out and resets the sector....even removing the players ship in the process. I managed to mitigate most of it in 1.6 but that's not released yet....standby let me see if i can find one of the old logs...bah dont have any for if i remove the scripts that add the menu to the carrier....but i do have one for the script loader

 

Well, if you just outright remove the script loader

http://pastebin.com/xs89nHKT

 

EDIT: that being said, part of it was my fault accidentally binding the script to NPC ships as well...because I'm and idiot =P

 

 

[koonschi]

That's a legit error created by the server since it expects all scripts to be there. It's basically there to show me immediately when something is wrong.

[Nexus]

That's a legit error created by the server since it expects all scripts to be there. It's basically there to show me immediately when something is wrong.

Yup, that's what happens if you don't unbind scriptloader.lua from player properly...if i remove carrier commands it just causes the sector to reset..and in some cases the sector vanishes completely your ship included but alas i have gotten around that now.

[Cypher]

I really would like to see a standardized way, the changing of core files by different mods and people copying and pasting in new mods without really understanding what they are doing is going to cause problems.

 

I don't think you can expect most of the players to be able to manually check mod compatibility nor expect them to be able to use the mod patcher tools that are popping up already. All they will most likely end up doing is break their game and experience.

 

I like the folder idea so each mod would get its own folder, if it would have a failsafe for when this folder is no longer exisiting and the mod gets removed from wherever it was referenced without causing server crashes.

[koonschi]

The folder idea seams reasonable, yeah.

[Cypher]

I think it would help out a lot as i mentioned above, it encapsulates mods better in that way making it easier and less problematic to remove any of them. I know it's work that you would need to do ofcourse don't get me wrong :P

[LoSboccacc]

pitching in: the way to go is for mod to register themselves to listen to game events, so that none needs to drop code on common parts.

 

also all stuff like crew number spawned per station etc should reside in a dictionary somewhere, so that mod can listen to 'beforeStart' event and change the values without having to change files.

[Cypher]

What i'm currently doing is writing my mods and (temporarily) add by a command. I'm having problems with my LUA skills being as rusty as can be, plus i've not worked a whole lot with OOP with lua. I'm currently trying to make changes to a station UI without even touching the script. I've just started on it so can't say if i can enough or not, as i said my Lua skills are too rusty to be able to look at the station script and figure out if i can or cannot access its window