SQLite
 

SQLite is a serverside only update that should have been added months ago, why don't we have it?

Though I've just about lost all interest in scripting due to lack of updates.

SQLite would be a great help when it comes to data storage. It's more efficient than flat-file storage, and doesn't have all the problems that flat-file systems run in to (and isn't a pain to script!).

There are tutorials for SQLite integration with Torque on the GarageGames resources web site. I would definitely prefer this being priority number one more than anything else.

Fun facts about the mud account files on Valikorlia:

Total count: 9,149
Total file contents size: 6.27 MB
Total file size including metadata (on Vista): 36.1 MB
And I compressed them into a 180 kB 7-Zip archive.

Would love to be using a database instead.

Ok added SQLite support, when using the requestsql-commands then the default database is now a sqlite database

Right now it's experimental (not checking for flooding etc.) so I can enable it on request.
To build the query string use string.escape() to avoid SQL injection.
The full list of supported SQL statements is available at http://www.sqlite.org/lang.html

afk

Fantastic. I know a lot of people have been waiting for this :D

I can't wait to try it out.

I'd like it enabled for Valikorlia and the Testbed Server.

So where is the database file stored? I would like to be able to back that up easily.

Also: what is the boolean value for, what are the parameters for requestsql2(), and what are the details on the object that the two functions return.

For new commands, couldn't you add a small description and what the parameters are? It wouldn't take that long :\

The boolean in requestsql() is probably for whether or not you want the SQL commands to return a value.

I suppose I should have said that I already had a good idea.

Edit: After looking up SQLite documentation, I have a few more suggestions.

I suggest instead of having the global requestsql() functions that instead there be an sqlopen(str filename) function that returns an sql database object that can be used to execute sql queries and such. This is so there can be multiple databases opened simultaneously.I don't want the whole server to use a single database, it's too easy for someone to just drop all the tables with a single command. I also don't think it would be too much trouble to add a waitforcompleted() function to the request object, which would return immediately if the request is completed or wait until it is completed.

Along with this, I noticed that before I have been unable to load files from disk during the onInitialized() in the Control-NPC, I'd like that fixed so there is no problem opening databases the instant the server starts.

The requestsql2() command lets you specify which configured database you are using. That way you can easily switch between sqlite or mysql. The database configuration is right now not accessible by server staff though.

requestsql() for sqlite databases is right now synchronous, but it can be possible later to also do it asynchronous like for mysql. In the example I have already shown how to wait for the command to complete:

It would be best to write your own wrapper class to do database commands (like getsql() and setsql()).

Update: the new npcserver is now installed on the private/hosted playerworld computers, restart it to try.

Did you enable SQL for Valikorlia like I asked? And I would like it if you could take a look at what I suggested in my previous post.

I'm also not going to be using SQL unless I can back the whole database up at least twice a day.

Pardon my ignorance, but what is the advantage of using a database (SQLite) to store information as opposed to text files or database NPCs?

You can easier make queries on the whole dataset, e.g. get all accounts with more than 100 hours online time (if you would have stored that information in the database).
Also you can access information independent from if the player is online or not, so it could be used on servers that currently store special additional attributes in files.

*Simmering*

Great new addition. And while we are talking about great new addition, could you please add Lua as a secondary, optional scripting language to Graal? I know this would be time consuming and lots of work/effort, but it would be so much worth it. Torque Engine Script (or whatever it is called) is a pile of poo compared to Lua. Lua is rated the fasted scripting language availabe.

Graal's scripting language was made by Stefan, it has nothing to do with Torque (other than the special GUI control syntax), it is merely compliant with some standard that I forget the name of.

Lua would be an improvement because it is a stable and well developed, and compact language. Of course we can't simply use Lua as it is because with Graal, security needs to be considered so Stefan would have to modify Lua to add security measures before he could use it.

It's too much work at this point.

I'm also obligated to say that I think Stackless Python is a better choice than Lua, but would be harder to modify for Graal-level security I think.

As far as I know, GScript2 is Torque Engine Script with quite a lot of Graal modifications to it. Correct me if I'm wrong though.

You are wrong, GScript2 is GScript2, its syntax is just made to conform to a standard.

Inverness is correct. (http://graal.net/index.php/TorqueScript) GS2 just borrows from its syntax.

And no, SQLite has not been added to Valikorlia... and 2 database backups per day is a bit unnecessary.

I see. Good to know, thanks.

Backups as a whole seem to be unnecessary to you, since you haven't made any backups yourself :). Though, I think once per day is a good interval.

No need to get snippy, Inverness. I don't back anything up because downloading files more than 1mb crashes my RC, 9 times out of 10.

In the rest of the scripting world, the number of times you back up data is proportionate to when the most changes occur (if data is updated weekly, there's no need to make daily backups), and sometimes how likely it is that you will need to restore that data (if you're not concerned with frequent server crashes). The only times Valikorlia has ever needed to restore back ups are when a staff member deletes everything in a hissy fit, and when the NPC server crashes. The first one happens more often than the latter, and the latter is only a problem because we're not using SQLite databases.

If you want to back-up all the information twice a day, or once a day, go ahead. I don't see the need for such paranoia, and am fine with weekly backups.

Excuses, I sure haven't heard about this problem of yours so I guess you didn't think it important enough to ask about.
Whether or not you are fine with weekly backups is irrelevant. You are staff and can easily manipulate your inventory, players can't and it's much easier to back up their inventories every day than to traverse logs to find out how their inventory has changed since the last backup in the event of a problem.

What can you do about it?
The point I made was that the likelihood of that being anything close to common occurrence is little to none. Let me break it down even further: the main reason why the NPC server has crashed is because of errors when using the INI file system.

Like I said, though. If you want back everything up that often, go ahead. I'm sure you could even script something that would do it automatically; some GS2 equivalent to a cron job.

EDIT: Could the new NPC server please be put on Valikorlia, already? :(

Ok the new npcserver has been uploaded to all machines, need to be restarted manually though.

Ok, first, what folder is the database file in? I would like if the one on Valikorlia could be placed somewhere visible to RC so it can be backed up easily. I don't feel too comfortable using it otherwise considering everything could be deleted with little effort.

Second, if the table has a column named "name" or like some other static variable then the row object has two variables named "name" in it, I was thinking that when setting the variables in the row object, if it conflicts with static variables then append an underscore.

The database is databases/main.db
Use "as" for renaming columns of the result. You can also access the result as array, like rows[][]

Ah good, thanks.

Could you perhaps make a function for converting arrays into a format suitable for SQL statements? Basically making it into a large string, using single quotes around strings, and escaping the single quotes in the string. I figure that would be better if not done by script.

And, I'd also like a server option for reporting SQLite errors and such on RC.

This is the equivalent script:

Well for errors I gave the suggestion to write a wrapper function:

For escaping use the string.escape() function.

I don't believe that is the same thing as what I'm talking about. For example, if a have a graal string "It's alive!" and I want to stick it in as a string value in an SQL statement string then it needs to be changed to "'It''s alive!'". the SQL strings have the single quotes around them plus any single quotes inside the string need to have a second single quote after it to be escaped.

Also, I don't think you specified what the boolean value was for in requestsql() and requestsql2().

It's in /scripthelp now:

Is there a way to create 'sub-tables'? It's been a while since I've using any SQL, and the one I did use was MySQL.

What I'm trying to do is create a sub-table of 'shops' to store item data.
However, I get the following error:

Just in general a 'sub-table' in a ERD is just 2 tables in a parent-child relationship. The child will always hold the foreign key to point towards the parent, not entirely sure why you are getting an error.

Sorry, my help was fairly useless.

Inverness, string.escape() is made for that. It escapes ' and \ according to the SQL standard / pascal.

Does the following link give any valuable information?

http://sqlite.org/omitted.html

More specifically, this bit on foreign-key constraints:

As for your error? No, not really, at least not that I can see. The fact that SQLite doesn't support referential integrity is surprising I'd say. Also no outer joins (granted, that can be accomplished in other ways.)

Using requestsql2() pointing to the "sqlite_master" database results in no error. However, when trying to access the data, nothing is outputted.
The execute() command is a simple error checker:

sqlite_master is a table not a database. Stefan said in an earlier post that requestsql2() is for choosing between mysql and sqlite databases, so at the moment it is useless to you and you shouldn't use it.

select tbl_name from sqlite_master where type='table' order by tbl_name

That would list all tables in the SQLite database.