libs/datafile.git
6 months agoUse automatic base detection when parsing integers master
Mikko Rasa [Wed, 21 Feb 2018 17:30:40 +0000 (19:30 +0200)]
Use automatic base detection when parsing integers

20 months agoAdd a missing include
Mikko Rasa [Fri, 6 Jan 2017 02:54:59 +0000 (04:54 +0200)]
Add a missing include

22 months agoAdd a source class for builtin data
Mikko Rasa [Tue, 15 Nov 2016 15:57:21 +0000 (17:57 +0200)]
Add a source class for builtin data

22 months agoProvide access to the keyword of the current statement
Mikko Rasa [Sat, 5 Nov 2016 07:47:56 +0000 (09:47 +0200)]
Provide access to the keyword of the current statement

22 months agoReplace files from earlier packs with those from the newly added one
Mikko Rasa [Sat, 29 Oct 2016 16:45:40 +0000 (19:45 +0300)]
Replace files from earlier packs with those from the newly added one

22 months agoPrioritize matching suffixes when looking for a type
Mikko Rasa [Wed, 26 Oct 2016 21:12:46 +0000 (00:12 +0300)]
Prioritize matching suffixes when looking for a type

2 years agoMerge remote-tracking branch 'origin/master'
Mikko Rasa [Mon, 19 Sep 2016 14:00:06 +0000 (17:00 +0300)]
Merge remote-tracking branch 'origin/master'

2 years agoMove all exception classes to except.h, for real this time
Mikko Rasa [Mon, 19 Sep 2016 13:45:15 +0000 (16:45 +0300)]
Move all exception classes to except.h, for real this time

2 years agoImplement a parser mode for JSON files
Mikko Rasa [Fri, 2 Sep 2016 12:34:48 +0000 (15:34 +0300)]
Implement a parser mode for JSON files

2 years agoMove token-to-argument conversion to Statement
Mikko Rasa [Fri, 2 Sep 2016 12:34:15 +0000 (15:34 +0300)]
Move token-to-argument conversion to Statement

2 years agoSome refactoring of TextParser logic
Mikko Rasa [Fri, 2 Sep 2016 11:44:08 +0000 (14:44 +0300)]
Some refactoring of TextParser logic

2 years agoMove all exception classes to a common header
Mikko Rasa [Fri, 2 Sep 2016 11:41:49 +0000 (14:41 +0300)]
Move all exception classes to a common header

2 years agoDon't use deprecated typedefs
Mikko Rasa [Sun, 31 Jan 2016 15:37:38 +0000 (17:37 +0200)]
Don't use deprecated typedefs

2 years agoStore offsets and sizes in PackSource as 64-bit ints
Mikko Rasa [Mon, 7 Dec 2015 19:01:13 +0000 (21:01 +0200)]
Store offsets and sizes in PackSource as 64-bit ints

2 years agoAdd a getter for fallback in Collection
Mikko Rasa [Tue, 24 Nov 2015 20:19:06 +0000 (22:19 +0200)]
Add a getter for fallback in Collection

2 years agoUse const sources in Collection
Mikko Rasa [Wed, 18 Nov 2015 20:45:18 +0000 (22:45 +0200)]
Use const sources in Collection

The CollectionSource interface is entirely const anyway.

2 years agoChain get_status to fallback collection if one is defined
Mikko Rasa [Sat, 14 Nov 2015 17:05:03 +0000 (19:05 +0200)]
Chain get_status to fallback collection if one is defined

2 years agoRemove the loaded flag from PackSource files
Mikko Rasa [Fri, 13 Nov 2015 08:53:35 +0000 (10:53 +0200)]
Remove the loaded flag from PackSource files

It seems to be a relic from ancient times.  It also causes problems in
certain situations where it's desirable to remove and reload items.

3 years agoAdd an unpack option to the data tool
Mikko Rasa [Mon, 7 Sep 2015 09:26:58 +0000 (12:26 +0300)]
Add an unpack option to the data tool

3 years agoPlug a memory leak
Mikko Rasa [Sat, 7 Mar 2015 02:07:09 +0000 (04:07 +0200)]
Plug a memory leak

3 years agoAccept int-to-float conversion in array signature
Mikko Rasa [Fri, 19 Dec 2014 03:34:44 +0000 (05:34 +0200)]
Accept int-to-float conversion in array signature

3 years agoAdd version of add_pack_io with filter
Mikko Rasa [Mon, 10 Nov 2014 17:47:56 +0000 (19:47 +0200)]
Add version of add_pack_io with filter

3 years agoSupport adding packs from IO::Seekable objects
Mikko Rasa [Sun, 19 Oct 2014 17:33:05 +0000 (20:33 +0300)]
Support adding packs from IO::Seekable objects

This allows using any seekable objects as packs, such as Android assets.

3 years agoProvide help for datatool's command line options
Mikko Rasa [Sun, 19 Oct 2014 13:00:27 +0000 (16:00 +0300)]
Provide help for datatool's command line options

Also use add_argument instead of the deprecated get_args.

3 years agoMore resiliece against certain corner cases when direct loading
Mikko Rasa [Sun, 19 Oct 2014 08:58:00 +0000 (11:58 +0300)]
More resiliece against certain corner cases when direct loading

An empty binary file terminated with __end was causing problems as the
very first peek call would return null, causing Loader to fall back to
indirect loading.  However, the __end was already processed by peek, so
the parse call would fail.

3 years agoMake Loader a virtual base of ObjectLoader to permit diamond inheritance
Mikko Rasa [Wed, 15 Oct 2014 18:22:07 +0000 (21:22 +0300)]
Make Loader a virtual base of ObjectLoader to permit diamond inheritance

Useful when creating loader classes through composition.

3 years agoFix a class/struct inconsistency
Mikko Rasa [Sun, 5 Oct 2014 18:21:51 +0000 (21:21 +0300)]
Fix a class/struct inconsistency

3 years agoRequire sigc++-2.0 since PackSource uses it
Mikko Rasa [Sun, 5 Oct 2014 11:54:14 +0000 (14:54 +0300)]
Require sigc++-2.0 since PackSource uses it

The signal connection may be refactored into libmspcore in the future,
but it's required here for now.

3 years agoAdd a flag to control replacement of DirectorySource entries
Mikko Rasa [Sun, 5 Oct 2014 01:50:48 +0000 (04:50 +0300)]
Add a flag to control replacement of DirectorySource entries

4 years agoUse dynamic_cast to support Collection as virtual base class
Mikko Rasa [Sun, 20 Apr 2014 09:53:33 +0000 (12:53 +0300)]
Use dynamic_cast to support Collection as virtual base class

4 years agoFix logic problems when ignoring a statement in the middle of direct load
Mikko Rasa [Thu, 19 Dec 2013 21:58:53 +0000 (23:58 +0200)]
Fix logic problems when ignoring a statement in the middle of direct load

Each recursive call to parse() was subtracting one from sub_remaining,
causing it to get out of sync with the actual statement hierarchy.

4 years agoDetect eof from the parser, not input file
Mikko Rasa [Tue, 17 Dec 2013 01:01:06 +0000 (03:01 +0200)]
Detect eof from the parser, not input file

Some files have an embedded logical eof, so waiting for the end of the
physical file would (and did) cause an infinite loop.

4 years agoExpose raw open functionality from Collection
Mikko Rasa [Sat, 12 Oct 2013 17:10:45 +0000 (20:10 +0300)]
Expose raw open functionality from Collection

There are several scenarios where streaming or asynchronous loading is
useful.

4 years agoInvoke creators from get_list
Mikko Rasa [Sat, 12 Oct 2013 15:55:33 +0000 (18:55 +0300)]
Invoke creators from get_list

4 years agoAvoid duplicate names if an item is available from multiple sources
Mikko Rasa [Sat, 12 Oct 2013 15:21:30 +0000 (18:21 +0300)]
Avoid duplicate names if an item is available from multiple sources

5 years agoExport file metadata from PackSource
Mikko Rasa [Thu, 5 Sep 2013 14:53:35 +0000 (17:53 +0300)]
Export file metadata from PackSource

5 years agoDon't forget to initialize the control statement flag
Mikko Rasa [Mon, 19 Aug 2013 14:00:11 +0000 (17:00 +0300)]
Don't forget to initialize the control statement flag

5 years agoMore efficient way of loading binary files
Mikko Rasa [Mon, 5 Aug 2013 11:56:26 +0000 (14:56 +0300)]
More efficient way of loading binary files

The Variant array for storing arguments in Statement is too slow.  Bypass
it by passing args directly to the action in a sort of ad-hoc struct.

5 years agoMove newline eating out of BinaryParser
Mikko Rasa [Mon, 5 Aug 2013 09:32:43 +0000 (12:32 +0300)]
Move newline eating out of BinaryParser

It's really a feature of the mode change logic, not the binary mode
itself.

5 years agoRestructure control statement processing
Mikko Rasa [Mon, 5 Aug 2013 09:19:31 +0000 (12:19 +0300)]
Restructure control statement processing

Binary format control statements can no longer occur as substatements, but
that was an accidental feature at best.

5 years agoReplace local RAII set utility with one from mspcore
Mikko Rasa [Sat, 3 Aug 2013 12:24:01 +0000 (15:24 +0300)]
Replace local RAII set utility with one from mspcore

5 years agoUse a common StatementKey structure for Loader and BinaryParser/Writer
Mikko Rasa [Sat, 3 Aug 2013 12:14:11 +0000 (15:14 +0300)]
Use a common StatementKey structure for Loader and BinaryParser/Writer

5 years agoConsider full statement signature when dealing with auxiliary loaders
Mikko Rasa [Sun, 16 Jun 2013 08:25:58 +0000 (11:25 +0300)]
Consider full statement signature when dealing with auxiliary loaders

5 years agoAdd a way to combine multiple loaders
Mikko Rasa [Sat, 15 Jun 2013 20:22:23 +0000 (23:22 +0300)]
Add a way to combine multiple loaders

5 years agoAdd a two-argument constructor to DerivedObjectLoader
Mikko Rasa [Sat, 15 Jun 2013 20:10:12 +0000 (23:10 +0300)]
Add a two-argument constructor to DerivedObjectLoader

5 years agoUse correct keyword in forward declaration
Mikko Rasa [Fri, 24 May 2013 14:47:03 +0000 (17:47 +0300)]
Use correct keyword in forward declaration

clang isn't happy if a struct is forward declared as a class, even though
they are semantically equivalent in that context.

5 years agoAdd accessors for data_error members
Mikko Rasa [Fri, 24 May 2013 14:46:20 +0000 (17:46 +0300)]
Add accessors for data_error members

5 years agoAdd ability to put raw files in a pack
Mikko Rasa [Sat, 13 Apr 2013 12:57:11 +0000 (15:57 +0300)]
Add ability to put raw files in a pack

5 years agoMiscellaneous cleanups
Mikko Rasa [Tue, 26 Mar 2013 20:37:37 +0000 (22:37 +0200)]
Miscellaneous cleanups

5 years agoUse the Sfinae helper struct from mspcore
Mikko Rasa [Tue, 26 Mar 2013 20:37:08 +0000 (22:37 +0200)]
Use the Sfinae helper struct from mspcore

5 years agoComment improvements
Mikko Rasa [Tue, 26 Mar 2013 20:36:17 +0000 (22:36 +0200)]
Comment improvements

5 years agoReturn null instead of throwing if a file is not found in sources
Mikko Rasa [Tue, 26 Mar 2013 20:15:19 +0000 (22:15 +0200)]
Return null instead of throwing if a file is not found in sources

The primary use case for open_from_sources is for creator functions to
open raw files.  This change makes it easier to gracefully skip a non-
existent file and carry on with other ways of loading the item.

5 years agoAdd a virtual destructor to CollectionSource
Mikko Rasa [Wed, 16 Jan 2013 14:05:38 +0000 (16:05 +0200)]
Add a virtual destructor to CollectionSource

5 years agoAdd an API to open files from a collection's sources
Mikko Rasa [Wed, 16 Jan 2013 14:01:28 +0000 (16:01 +0200)]
Add an API to open files from a collection's sources

5 years agoUse IO::Slice to constrain access to logical files
Mikko Rasa [Sat, 5 Jan 2013 11:03:59 +0000 (13:03 +0200)]
Use IO::Slice to constrain access to logical files

5 years agoPass mode to ZlibCompressed
Mikko Rasa [Sat, 5 Jan 2013 09:40:42 +0000 (11:40 +0200)]
Pass mode to ZlibCompressed

This makes creating compressed packs work.  The temporary file is opened
with M_RDWR because we want to read the data back from the same handle, so
ZlibCompressed can't autodetect the mode.

5 years agoAdd a method to filter the files loaded from a pack
Mikko Rasa [Fri, 4 Jan 2013 17:29:58 +0000 (19:29 +0200)]
Add a method to filter the files loaded from a pack

5 years agoAdd support for fallback collections
Mikko Rasa [Fri, 7 Dec 2012 10:28:15 +0000 (12:28 +0200)]
Add support for fallback collections

5 years agoSome minor fixes in Collection
Mikko Rasa [Thu, 6 Dec 2012 21:44:11 +0000 (23:44 +0200)]
Some minor fixes in Collection

Move RemoveConst invovations close to the public interface to reduce the
number of template function instantiations.

Correct access specifiers.

5 years agoRework the list and containment queries
Mikko Rasa [Wed, 5 Dec 2012 19:16:00 +0000 (21:16 +0200)]
Rework the list and containment queries

Most of the implementations are now in .cpp, and they should deal better
with base classes.

5 years agoAdd some new test cases for Collection
Mikko Rasa [Wed, 5 Dec 2012 16:57:28 +0000 (18:57 +0200)]
Add some new test cases for Collection

Failures will be fixed a bit later.

5 years agoRestore the check to avoid deletion if an add call fails
Mikko Rasa [Tue, 4 Dec 2012 10:28:24 +0000 (12:28 +0200)]
Restore the check to avoid deletion if an add call fails

It was lost to a careless rewrite in b1bc256.

5 years agoUpdate Collection test cases with source and base class changes
Mikko Rasa [Tue, 4 Dec 2012 09:08:11 +0000 (11:08 +0200)]
Update Collection test cases with source and base class changes

5 years agoMore flexible system for handling base classes in Collection
Mikko Rasa [Tue, 4 Dec 2012 08:32:43 +0000 (10:32 +0200)]
More flexible system for handling base classes in Collection

Items are now stored with their original type, but can be retrieved with
a base class as well.

5 years agoTurn internal type helpers into structs
Mikko Rasa [Mon, 3 Dec 2012 21:58:06 +0000 (23:58 +0200)]
Turn internal type helpers into structs

Eliminating access specifiers and dummy constructors reduces clutter, and
in most cases pure virtual functions prevent instantiation anyway.  Since
these reside in private or protected sections of other classes, risk of
misuse is negligible.

5 years agoMove most of Collection::get implementation to collection.cpp
Mikko Rasa [Mon, 3 Dec 2012 21:20:02 +0000 (23:20 +0200)]
Move most of Collection::get implementation to collection.cpp

5 years agoUse an Fmt object to store the float format
Mikko Rasa [Sat, 1 Dec 2012 09:45:04 +0000 (11:45 +0200)]
Use an Fmt object to store the float format

5 years agoAdd the target type to lexical_cast invocations
Mikko Rasa [Sat, 1 Dec 2012 09:32:19 +0000 (11:32 +0200)]
Add the target type to lexical_cast invocations

5 years agoCache filenames in DirectorySource
Mikko Rasa [Thu, 15 Nov 2012 21:25:59 +0000 (23:25 +0200)]
Cache filenames in DirectorySource

5 years agoSimplify DirectorySource path management
Mikko Rasa [Thu, 15 Nov 2012 21:03:43 +0000 (23:03 +0200)]
Simplify DirectorySource path management

5 years agoRedesign automatic object loading
Mikko Rasa [Thu, 15 Nov 2012 19:04:09 +0000 (21:04 +0200)]
Redesign automatic object loading

Replace the Collection subclasses with a new concept of CollectionSources.
This permits more than one way of loading objects.

5 years agoFix comment handling
Mikko Rasa [Sun, 4 Nov 2012 19:15:40 +0000 (21:15 +0200)]
Fix comment handling

A comment start sequence inside another comment should not change the
comment type.

5 years agoAdd a test case for comments
Mikko Rasa [Sun, 4 Nov 2012 19:15:02 +0000 (21:15 +0200)]
Add a test case for comments

5 years agoAdd an intelligent packed collection class
Mikko Rasa [Fri, 28 Sep 2012 23:34:47 +0000 (02:34 +0300)]
Add an intelligent packed collection class

5 years agoThrow an exception if a future object couldn't be created
Mikko Rasa [Fri, 28 Sep 2012 23:01:34 +0000 (02:01 +0300)]
Throw an exception if a future object couldn't be created

5 years agoPreserve source declarations when transferring a collection
Mikko Rasa [Fri, 28 Sep 2012 22:55:03 +0000 (01:55 +0300)]
Preserve source declarations when transferring a collection

Optionally print all internal statements (useful to see what's happening
inside a binary datafile).

5 years agoAdd a flag to return also internal statements
Mikko Rasa [Fri, 28 Sep 2012 22:54:06 +0000 (01:54 +0300)]
Add a flag to return also internal statements

5 years agoAdd a special keyword to terminate parsing
Mikko Rasa [Fri, 28 Sep 2012 09:11:58 +0000 (12:11 +0300)]
Add a special keyword to terminate parsing

5 years agoImprove the API for future objects
Mikko Rasa [Fri, 28 Sep 2012 09:10:23 +0000 (12:10 +0300)]
Improve the API for future objects

add_future is now documented as ignoring names that don't match any type,
and there's a new function to add future objects with a keyword.

5 years agoRemove the outdated except.h
Mikko Rasa [Fri, 28 Sep 2012 09:04:38 +0000 (12:04 +0300)]
Remove the outdated except.h

5 years agoAdd a shortcut for wrapping data for a collection file
Mikko Rasa [Thu, 27 Sep 2012 22:31:37 +0000 (01:31 +0300)]
Add a shortcut for wrapping data for a collection file

Collections are part of the library itself, so there's no reason to
require users to write all the boilerplate to do the wrapping.

5 years agoCreate writers dynamically and delete them before deleting output
Mikko Rasa [Thu, 27 Sep 2012 22:24:13 +0000 (01:24 +0300)]
Create writers dynamically and delete them before deleting output

This ensures that the IO objects are destroyed in the correct order, and
filters won't try to access deleted objects.

5 years agoRecognize char pointers as strings
Mikko Rasa [Thu, 27 Sep 2012 22:15:32 +0000 (01:15 +0300)]
Recognize char pointers as strings

statement.append("literal") was creating a symbol value, which was not
following the principle of least surprise.

5 years agoRestructure the tool and make it able to handle multiple input files
Mikko Rasa [Thu, 27 Sep 2012 16:23:58 +0000 (19:23 +0300)]
Restructure the tool and make it able to handle multiple input files

5 years agoSimplify loading collection items
Mikko Rasa [Thu, 27 Sep 2012 13:11:56 +0000 (16:11 +0300)]
Simplify loading collection items

The ItemLoader wrapper from DirectoryCollection can be used in the base
Collection::Loader as well.  This removes the need for the kludgy struct
Add and the need for two separate loader functions.  As an added bonus,
the ItemLoader can still be used in DirectoryCollection and future derived
classes as well, further reducing duplication.

5 years agoMore unit tests for Collection, including contains and future objects
Mikko Rasa [Wed, 26 Sep 2012 20:29:57 +0000 (23:29 +0300)]
More unit tests for Collection, including contains and future objects

5 years agoFix some minor mistakes
Mikko Rasa [Wed, 26 Sep 2012 20:26:54 +0000 (23:26 +0300)]
Fix some minor mistakes

5 years agoGive DirectoryCollection the ability to add files as future objects
Mikko Rasa [Wed, 26 Sep 2012 19:18:24 +0000 (22:18 +0300)]
Give DirectoryCollection the ability to add files as future objects

5 years agoMinor reorganization
Mikko Rasa [Wed, 26 Sep 2012 18:49:22 +0000 (21:49 +0300)]
Minor reorganization

5 years agoIntroduce the concept of future objects
Mikko Rasa [Wed, 26 Sep 2012 18:36:28 +0000 (21:36 +0300)]
Introduce the concept of future objects

5 years agoMake Collection::contains check for type
Mikko Rasa [Wed, 26 Sep 2012 16:04:33 +0000 (19:04 +0300)]
Make Collection::contains check for type

Checking item existence without type information is not particularly
useful, since there's no guarantee that the object can be used for its
intended purpose.

5 years agoImprove the documentation of Collection
Mikko Rasa [Wed, 26 Sep 2012 07:43:01 +0000 (10:43 +0300)]
Improve the documentation of Collection

6 years agoRevert "Convenience function for reporting errors in loaded data"
Mikko Rasa [Thu, 30 Aug 2012 08:35:53 +0000 (11:35 +0300)]
Revert "Convenience function for reporting errors in loaded data"

This reverts commit b0b9af7216560da2a46ea38fe2df959f4dfb126f.

Why did I even add this function.  The Loader catches any exceptions and
rethrows them as data_errors with the correct line number information.

6 years agoFix some whitespace errors
Mikko Rasa [Thu, 30 Aug 2012 08:35:37 +0000 (11:35 +0300)]
Fix some whitespace errors

6 years agoChange DerivedObjectLoader to take the base loader as template argument
Mikko Rasa [Tue, 21 Aug 2012 23:15:41 +0000 (02:15 +0300)]
Change DerivedObjectLoader to take the base loader as template argument

This makes it much more versatile, as it can now be used to derive from
a protected loader or one with a nonstandard name.

6 years agoProper loading of collection-enabled objects in DirectoryCollection
Mikko Rasa [Sat, 18 Aug 2012 21:16:50 +0000 (00:16 +0300)]
Proper loading of collection-enabled objects in DirectoryCollection

6 years agoMake DirectoryCollection::lookup_file available to derived classes
Mikko Rasa [Sat, 18 Aug 2012 21:16:13 +0000 (00:16 +0300)]
Make DirectoryCollection::lookup_file available to derived classes

6 years agoAdd support for compressed datafiles
Mikko Rasa [Fri, 3 Aug 2012 10:06:13 +0000 (13:06 +0300)]
Add support for compressed datafiles

6 years agoAdd an abstraction layer for output
Mikko Rasa [Fri, 3 Aug 2012 09:55:03 +0000 (12:55 +0300)]
Add an abstraction layer for output

6 years agoMore senseful validity checks for keyword and string definitions
Mikko Rasa [Thu, 2 Aug 2012 14:45:00 +0000 (17:45 +0300)]
More senseful validity checks for keyword and string definitions

I don't see how the original conditions could ever have been triggered,
given that arguments are parsed based on the signature, and those of the
built-in statements are fixed.