Graal Forums  

Go Back   Graal Forums > Development Forums > NPC Scripting > Code Gallery
FAQ Members List Calendar Today's Posts

Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 07-09-2010, 06:11 PM
cbk1994 cbk1994 is offline
the fake one
cbk1994's Avatar
Join Date: Mar 2003
Location: San Francisco
Posts: 10,718
cbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond repute
Send a message via AIM to cbk1994
GBall Creator (Automated Server Backup)

GBall Creator is a script to generate a "GBall package". This package can contain multiple files. Essentially, it works as a ZIP file but can be created serverside.

An example of why this is useful is that with one line of code you can backup a server's folder structure (selectively) — it can even be used to backup the accounts folder.

PHP Code:
function onCreated() {
  
this.version "1.0";     // eventually used in expander?
  
  
temp.mb 1048576;
  
this.maxWrite = (mb 2); // max to write at once; I find mb*9 is the absolute limit
                            // but below that is usually better to avoid flood alert
  
  
this.loopLimit 5000;    // the number of loops before a sleep -- must be less than
                            // this.maxlooplimit -- 5000 is good but don't expect
                            // the server to be playable while the files are added to
                            // the "to write" list (the first part which takes very little
                            // time, but will lock up the server)
  
  
this.delay 60 6;      // delay between trying to write, default 60*6 -- this seems not to
                            // cause flood alert, which, while protected against, can prevent
                            // other scripts on the server from functioning
}

/*
    The way folders work is that if I were to compress
    only "weapons/*", there would be a folder called
    "weapons" with all of the contents inside.
    
    If I were to compress "weapons/*" and "levels/world/*",
    there would be two folders: "weapons" and "world"
*/

public function GBall(descriptionoutputNamefolders) {
  
temp.lines null;
  
  for (
temp.folderName folders) {
    
temp.folder.loadFolder(folderName[0], folderName[1]);
    
temp.folderName[0].positions("/");
    
temp.pathToFolder folderName[0].substring(0p[p.size() - 1]);
    
temp.tokens pathToFolder.tokenize("/");
    
temp.bottomFolderName tokens[tokens.size() - 1];
    
    for (
temp.file folder) {
      
// add it to the package
      
temp.str.loadString(pathToFolder "/" file);
      
str "  " base64encode(bottomFolderName "/" file) @ " " base64encode(str);
      
lines.add(str);
      
      if (
this.loop()) {
        echo(
"GBall: Building file cache...");
      }
    }
    
    
this.loop();
  }
  
  
// don't save it all at once to save resources + avoid flood alert
  
temp.line 0;
  
temp.0;
  
  while (
line lines.size()) {
    
temp.chars 0;
    
temp.toWrite = {format("GBALL v%s (%s): %s"this.versionbase64encode(description), int(timevar2))};
    
    while (
chars this.maxWrite && line lines.size()) {
      
toWrite.add(lines[line]);
      
chars += lines[line].length();
      
line ++;
      
this.loop();
    }
    
    
temp.fileName outputName ".arc";
    
temp.hasTried false;
    
    while (! (
fileExists(fileName))) {
      if (
hasTried) {
        echo(
"GBall: Unable to write '" fileName "'; flood alert or bad permissions. Trying again after delay...");
        
sleep(this.delay);
      }
      
      
toWrite.saveLines(outputName ".arc"false);
      echo(
"GBall: Writing...");
      
      
hasTried true;
      
sleep(1); // sometimes it takes a minute for the file to "exist"
    
}
    
    
// sleep to avoid flood alert
    
if (line lines.size()) {
      
sleep(this.delay);
    }
    
    
++;
  }
  
  echo(
"GBall: Saved file (" outputName "x.arc)!");
}

function 
loop() {
  
this.count ++;
  
  if (
this.count >= this.loopLimit) {
    
this.count 0;
    
sleep(1);
    return 
true;
  }

An example of using this:
PHP Code:
// description, file prefix, list of folders
GBall("Backup of accounts""data/accountsbk_", {{"accounts/*"true}}); 
This will create a backup of the accounts folder, including the folder structure, and save it in several files (depending on how many accounts you have on the server) in the "data" folder. The reason for saving it in several files is that the max file size is 10 MB for script writing.

The best part of the script is that it is optimized to work on populated servers. Initially the script will load all files needed into memory. This is the only part that will cause the server to lock up. After this, the files will be written every few minutes to evade the flood alert. There is protection so that if the flood alert kicks in and prevents a file from being written it will wait and then attempt to write it after the flood alert has subsided. This stage (file writing) can take hours, depending on the settings you specify at the top of the script, but it will not lag the server.

Once the script has finished you will have files with a ".arc" extension. Download all of these files, then open the GBallExpander Java application included at the end of this post and select the first file. The application will expand the GBall files into the actual folder structure. This should not take long at all. You can also store the files in their GBall form and expand them later (you may wish to ZIP them to save file space).

The Java application which expands the GBall files works fine on Mac and Windows, and probably any other system with the JRE installed.




I should also mention that this script works fine to backup any type of file I've been able to throw at it, from text files to levels to images. Essentially you could backup the entire levels/* folder on your server.

I used this on Era last night to backup the accounts folder and it worked flawlessly, so it should work just about anywhere.

I feel like I've done a ****ty job explaining in this post, so feel free to ask questions.

I haven't yet received permission to post a link to the source code for the Java application so forum PM me if you want it (or decompile the JAR).
Attached Files
File Type: zip GBallExpander.jar.zip (4.1 KB, 1258 views)
__________________

Last edited by cbk1994; 07-09-2010 at 07:33 PM..
Reply With Quote
  #2  
Old 07-10-2010, 12:28 AM
maximus_asinus maximus_asinus is offline
RIP DarkCloud_PK
Join Date: Oct 2001
Location: Canada
Posts: 3,745
maximus_asinus has a reputation beyond reputemaximus_asinus has a reputation beyond reputemaximus_asinus has a reputation beyond reputemaximus_asinus has a reputation beyond reputemaximus_asinus has a reputation beyond reputemaximus_asinus has a reputation beyond reputemaximus_asinus has a reputation beyond reputemaximus_asinus has a reputation beyond reputemaximus_asinus has a reputation beyond reputemaximus_asinus has a reputation beyond repute
This would be very handy to have since Stefan stopped backing up playerworlds. Very nice. Rep++.
__________________
Save Classic!
Reply With Quote
  #3  
Old 07-13-2010, 06:00 PM
sssssssssss sssssssssss is offline
Cyril Rain
sssssssssss's Avatar
Join Date: May 2003
Location: Texas, USA
Posts: 1,134
sssssssssss will become famous soon enough
Should be sticky. Rep+.
__________________
Cyril Rain
Creator and leader of SLX
Admin of Elysium
Elysium's Facebook Page: http://facebook.com/GraalOnlineElysium
Graal Forum Thread: http://forums.graalonline.com...
Graalians Thread: http://www.graalians.com...


Reply With Quote
  #4  
Old 07-14-2010, 05:46 AM
Seeya Seeya is offline
ω
Seeya's Avatar
Join Date: Jul 2007
Location: Seminole, FL
Posts: 1,903
Seeya is a splendid one to beholdSeeya is a splendid one to beholdSeeya is a splendid one to beholdSeeya is a splendid one to beholdSeeya is a splendid one to behold
Send a message via AIM to Seeya
now i have gballs all over my dekstop, thanks
__________________
Kale Vimes

Reply With Quote
  #5  
Old 07-14-2010, 04:14 PM
WhiteDragon WhiteDragon is offline
Banned
Join Date: Feb 2007
Posts: 1,002
WhiteDragon is a splendid one to beholdWhiteDragon is a splendid one to beholdWhiteDragon is a splendid one to beholdWhiteDragon is a splendid one to beholdWhiteDragon is a splendid one to behold
Doesn't this involve having all the files you are putting in the GBall in memory at once? You should make it so it flushes the files in memory to the file every once in awhile (otherwise the machine will start swapping and slow down to a halt).
Reply With Quote
  #6  
Old 07-14-2010, 04:16 PM
Crono Crono is offline
:pluffy:
Join Date: Feb 2002
Location: Sweden
Posts: 20,000
Crono has a reputation beyond reputeCrono has a reputation beyond reputeCrono has a reputation beyond reputeCrono has a reputation beyond reputeCrono has a reputation beyond reputeCrono has a reputation beyond reputeCrono has a reputation beyond reputeCrono has a reputation beyond reputeCrono has a reputation beyond reputeCrono has a reputation beyond reputeCrono has a reputation beyond repute
lol, players tackling problems cyberjoueurs should be doing in the first place. nice though

edit- sign your name in neg reps ffs
__________________

Last edited by Crono; 07-14-2010 at 11:29 PM..
Reply With Quote
  #7  
Old 07-14-2010, 07:02 PM
cbk1994 cbk1994 is offline
the fake one
cbk1994's Avatar
Join Date: Mar 2003
Location: San Francisco
Posts: 10,718
cbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond repute
Send a message via AIM to cbk1994
Quote:
Originally Posted by WhiteDragon View Post
Doesn't this involve having all the files you are putting in the GBall in memory at once? You should make it so it flushes the files in memory to the file every once in awhile (otherwise the machine will start swapping and slow down to a halt).
The problem is that I can't. If I write to files any more than it is already the flood alert is hit and it's impossible to write the files for saving memory or for saving the GBall files. All that it does is load files, then write them to GBall files (the same as saving to files).

Granted, I can remove files which have been written already from the memory. I'll release a new version soon which does this (and a few other optimizations I've added).
__________________
Reply With Quote
  #8  
Old 07-28-2010, 07:53 PM
cbk1994 cbk1994 is offline
the fake one
cbk1994's Avatar
Join Date: Mar 2003
Location: San Francisco
Posts: 10,718
cbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond repute
Send a message via AIM to cbk1994
Releasing a new, much improved version. It now drops lines which have been written out of memory, which can be very important on populated servers. It also only checks the line count once, rather than before when it was checked each time a file was written (this was actually causing the most slowdown).

It also now provides better progress indications (such as the percent it is finished with writing) in echoes.

PHP Code:
// default setups
enum {
  
FOLDER_NPCS 10 // npcs/* minus npclocalnpc*
}

function 
onCreated() { 
  
this.version "1.2";     // eventually used in expander? 
   
  
temp.mb 1048576
  
this.maxWrite = (mb 4); // max to write at once; I find mb*9 is the absolute limit 
                            // but below that is usually better to avoid flood alert 
   
  
this.loopLimit 5000;    // the number of loops before a sleep -- must be less than 
                            // this.maxlooplimit -- 5000 is good but don't expect 
                            // the server to be playable while the files are added to 
                            // the "to write" list (the first part which takes very little 
                            // time, but will lock up the server) 
   
  
this.delay 60 6;      // delay between trying to write, default 60*6 -- this seems not to 
                            // cause flood alert, which, while protected against, can prevent 
                            // other scripts on the server from functioning 


/* 
    The way folders work is that if I were to compress 
    only "weapons/*", there would be a folder called 
    "weapons" with all of the contents inside. 
     
    If I were to compress "weapons/*" and "levels/world/*", 
    there would be two folders: "weapons" and "world" 
*/ 

public function GBall(descriptionoutputNamefolders) { 
  
temp.lines null
  
temp.c2 0;
  
  for (
temp.folderName folders) {
    
c2 ++;
    
temp.defSetup null;
    
    if (
folderName == FOLDER_NPCS) {
      
folderName = {"npcs/*"false};
      
defSetup FOLDER_NPCS;
    }
    
    
temp.folder.loadFolder(folderName[0], folderName[1]); 
    
temp.folderName[0].positions("/"); 
    
temp.pathToFolder folderName[0].substring(0p[p.size() - 1]); 
    
temp.tokens pathToFolder.tokenize("/"); 
    
temp.bottomFolderName tokens[tokens.size() - 1]; 
    
temp.0;
    
temp.fSize folder.size();
    
    for (
temp.file folder) {
      
temp.++;
      
      if (
this.loop()) { 
        echo(
"GBall: Building file cache... " int(fSize 100) @ "% (folder " temp.c2  "/" folders.size() @ ")"); 
      } 
      
      if (
defSetup == FOLDER_NPCS) {
        if (
file.starts("npclocalnpc")) {
          continue;
        }
      }
      
      
// add it to the package 
      
temp.str.loadString(pathToFolder "/" file); 
      
str "  " base64encode(bottomFolderName "/" file) @ " " base64encode(str); 
      
lines.add(str); 
    } 
     
    
this.loop(); 
  }
  
  
// don't save it all at once to save resources + avoid flood alert 
  
temp.line 0
  
temp.0
  
temp.lineSize lines.size();
   
  while (
line lineSize) {
    
temp.chars 0
    
temp.toWrite = {format("GBALL v%s (%s): %s"this.versionbase64encode(description), int(timevar2))}; 
    
    while (
chars this.maxWrite && line lineSize) { 
      
toWrite.add(lines[0]); 
      
chars += lines[0].length(); 
      
lines.delete(0);
      
line ++;
      
this.loop(); 
    }
     
    
temp.fileName outputName ".arc"
    
temp.hasTried false
     
    while (! (
fileExists(fileName))) { 
      if (
hasTried) { 
        echo(
"GBall: Unable to write '" fileName "'; flood alert or bad permissions. Trying again after delay..."); 
        
sleep(this.delay); 
      } 
       
      
toWrite.saveLines(outputName ".arc"false); 
      echo(
"GBall: Writing... " int(line lineSize 100) @ "%"); 
       
      
hasTried true
      
sleep(1); // sometimes it takes a minute for the file to "exist" 
    

     
    
// sleep to avoid flood alert 
    
if (line lineSize) { 
      
sleep(this.delay); 
    } 
     
    
++; 
  } 
   
  echo(
"GBall: Saved file (" outputName "x.arc)!"); 


function 
loop() { 
  
this.count ++; 
   
  if (
this.count >= this.loopLimit) { 
    
this.count 0
    
sleep(1); 
    return 
true
  } 

I've also added "folder configurations". This lets you do custom things for including files. I've added one, which is FOLDER_NPCS. This will back up all DB NPCs but not backup local NPCs (putnpc2).

Example usage:
PHP Code:
GBall("Backup of NC 7/28/10""data/2010_07_28_scripts_", {{"scripts/*"false}, {"weapons/*"false}, FOLDER_NPCS}); 
In addition, I'm releasing a new version of the expander (attached). It includes a bug fix for files which don't have an underscore before the number. It's also up to 30% faster due to some optimizations for folders with a lot of files.

The source code for the expander (approved by Skyld) can be found here.

(tip: combine this with Dylan's GraalCron for making daily NC backups)
Attached Files
File Type: zip GBallExpander.jar.zip (4.1 KB, 1127 views)
__________________
Reply With Quote
  #9  
Old 08-07-2010, 12:38 AM
sssssssssss sssssssssss is offline
Cyril Rain
sssssssssss's Avatar
Join Date: May 2003
Location: Texas, USA
Posts: 1,134
sssssssssss will become famous soon enough
I'm a dumb dumb, so where do you put this code (wnpc? dbnpc?) and how to call the script to go? RC chat?

See? I r dumb dumb.
__________________
Cyril Rain
Creator and leader of SLX
Admin of Elysium
Elysium's Facebook Page: http://facebook.com/GraalOnlineElysium
Graal Forum Thread: http://forums.graalonline.com...
Graalians Thread: http://www.graalians.com...


Reply With Quote
  #10  
Old 08-07-2010, 12:45 AM
cbk1994 cbk1994 is offline
the fake one
cbk1994's Avatar
Join Date: Mar 2003
Location: San Francisco
Posts: 10,718
cbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond repute
Send a message via AIM to cbk1994
Put it in a DB NPC and then just call the GBall function (see the examples in the first post). Make sure (npcserver) has read rights to the folders to backup and rw to the output folder.
__________________
Reply With Quote
  #11  
Old 08-07-2010, 01:09 AM
sssssssssss sssssssssss is offline
Cyril Rain
sssssssssss's Avatar
Join Date: May 2003
Location: Texas, USA
Posts: 1,134
sssssssssss will become famous soon enough
Lol, I was typing it in rc for some reason. xD
Where does it save to?
__________________
Cyril Rain
Creator and leader of SLX
Admin of Elysium
Elysium's Facebook Page: http://facebook.com/GraalOnlineElysium
Graal Forum Thread: http://forums.graalonline.com...
Graalians Thread: http://www.graalians.com...


Reply With Quote
  #12  
Old 08-07-2010, 01:18 AM
cbk1994 cbk1994 is offline
the fake one
cbk1994's Avatar
Join Date: Mar 2003
Location: San Francisco
Posts: 10,718
cbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond repute
Send a message via AIM to cbk1994
Quote:
Originally Posted by sssssssssss View Post
Lol, I was typing it in rc for some reason. xD
Where does it save to?
If you're doing this:
PHP Code:
GBall("Backup of accounts""data/accountsbk_", {{"accounts/*"true}}); 
It will save to the 'data' folder. You can change that in the second parameter. Make sure the npcserver has rw to the folder.
__________________
Reply With Quote
  #13  
Old 08-07-2010, 01:22 AM
sssssssssss sssssssssss is offline
Cyril Rain
sssssssssss's Avatar
Join Date: May 2003
Location: Texas, USA
Posts: 1,134
sssssssssss will become famous soon enough
ah, thanks. :O
__________________
Cyril Rain
Creator and leader of SLX
Admin of Elysium
Elysium's Facebook Page: http://facebook.com/GraalOnlineElysium
Graal Forum Thread: http://forums.graalonline.com...
Graalians Thread: http://www.graalians.com...


Reply With Quote
  #14  
Old 08-07-2010, 01:33 AM
sssssssssss sssssssssss is offline
Cyril Rain
sssssssssss's Avatar
Join Date: May 2003
Location: Texas, USA
Posts: 1,134
sssssssssss will become famous soon enough
Im using this.
PHP Code:
GBall("Backup of NC 7/28/10""data/2010_07_28_scripts_", {{"scripts/*"false}, {"weapons/*"false}, FOLDER_NPCS}); 
But its only backing up classes. For the hell of it I put the false's to true's. Did the same thing. NPC server has all rights.
__________________
Cyril Rain
Creator and leader of SLX
Admin of Elysium
Elysium's Facebook Page: http://facebook.com/GraalOnlineElysium
Graal Forum Thread: http://forums.graalonline.com...
Graalians Thread: http://www.graalians.com...


Reply With Quote
  #15  
Old 08-07-2010, 01:36 AM
cbk1994 cbk1994 is offline
the fake one
cbk1994's Avatar
Join Date: Mar 2003
Location: San Francisco
Posts: 10,718
cbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond reputecbk1994 has a reputation beyond repute
Send a message via AIM to cbk1994
Post the NPC-server rights?
__________________
Reply With Quote
Reply

Tags
archive, backup


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 08:21 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.
Copyright (C) 1998-2019 Toonslab All Rights Reserved.