libs/datafile.git
7 months agoSupport custom action maps in loader classes master
Mikko Rasa [Sun, 19 Apr 2020 14:30:36 +0000 (17:30 +0300)]
Support custom action maps in loader classes

This makes it possible to implement a persistent action map which is
only initialized once the first time a particular class is used.

7 months agoSupport loader functions with bound first argument
Mikko Rasa [Sun, 19 Apr 2020 14:24:55 +0000 (17:24 +0300)]
Support loader functions with bound first argument

This allows more flexibility in defining dynamic keywords.  Only
available with C++11.

7 months agoSupport N-ary loader functions if compiling in C++11 mode
Mikko Rasa [Sun, 19 Apr 2020 14:23:50 +0000 (17:23 +0300)]
Support N-ary loader functions if compiling in C++11 mode

7 months agoAllow non-loadable types in collections
Mikko Rasa [Sun, 19 Apr 2020 14:09:07 +0000 (17:09 +0300)]
Allow non-loadable types in collections

Custom collection classes may wish to provide objects of such types
through creator functions.

7 months agoUpdate a SFINAE construct
Mikko Rasa [Sun, 19 Apr 2020 13:24:11 +0000 (16:24 +0300)]
Update a SFINAE construct

7 months agoPrefer vector over list
Mikko Rasa [Sun, 19 Apr 2020 13:22:51 +0000 (16:22 +0300)]
Prefer vector over list

15 months agoThrow an exception if trying to load a nonexistent file
Mikko Rasa [Wed, 21 Aug 2019 22:06:52 +0000 (01:06 +0300)]
Throw an exception if trying to load a nonexistent file

Collection::open_raw returns a null pointer in this case so it must be
checked.

17 months agoAdd support for generating .cpp files for BuiltinSource
Mikko Rasa [Wed, 12 Jun 2019 16:43:28 +0000 (19:43 +0300)]
Add support for generating .cpp files for BuiltinSource

17 months agoUse IO::BufferedFile in the packer component of datatool
Mikko Rasa [Wed, 12 Jun 2019 16:42:25 +0000 (19:42 +0300)]
Use IO::BufferedFile in the packer component of datatool

17 months agoAdd a utility for registering types for loading
Mikko Rasa [Wed, 12 Jun 2019 10:13:22 +0000 (13:13 +0300)]
Add a utility for registering types for loading

Somewhat ironically Collection can't use this because it has much more
complex requirements.

17 months agoAllow modifying existing types in a collection
Mikko Rasa [Fri, 7 Jun 2019 17:27:17 +0000 (20:27 +0300)]
Allow modifying existing types in a collection

18 months agoAnnotate deprecated identifiers as such
Mikko Rasa [Mon, 3 Jun 2019 11:54:16 +0000 (14:54 +0300)]
Annotate deprecated identifiers as such

18 months agoUse a larger buffer size when transferring pack contents
Mikko Rasa [Mon, 3 Jun 2019 08:22:05 +0000 (11:22 +0300)]
Use a larger buffer size when transferring pack contents

18 months agoRespect the TMPDIR environment variable when creating data packs
Mikko Rasa [Mon, 3 Jun 2019 08:20:23 +0000 (11:20 +0300)]
Respect the TMPDIR environment variable when creating data packs

18 months agoPrefer exact type match when looking for Loader actions
Mikko Rasa [Mon, 3 Jun 2019 08:16:18 +0000 (11:16 +0300)]
Prefer exact type match when looking for Loader actions

Previosuly the first matching action in alphabetical signature order was
taken, which causes a float overload to be chosen for an int value even
if an int overload was also available.

18 months agoAdd a test case for Loader overload resolution
Mikko Rasa [Mon, 3 Jun 2019 08:15:11 +0000 (11:15 +0300)]
Add a test case for Loader overload resolution

18 months agoUpdate Collection terminology
Mikko Rasa [Mon, 3 Jun 2019 08:12:40 +0000 (11:12 +0300)]
Update Collection terminology

Mostly in test cases but there was one lingering reference to "future"
objects in the actual header as well.

18 months agoFix test cases so they compile again
Mikko Rasa [Mon, 3 Jun 2019 07:29:35 +0000 (10:29 +0300)]
Fix test cases so they compile again

18 months agoAdd more flexible versions of the load function
Mikko Rasa [Fri, 24 May 2019 17:25:39 +0000 (20:25 +0300)]
Add more flexible versions of the load function

Firstly there's a variadic version of the basic load function.  Secondly
and perhaps more importantly there are versions which can load an object
from a file stored in a collection.  All of these are only implemented
for C++11 for now, since supporting the old version seems increasingly
irrelevant.

18 months agoMove NeedsCollection into its own header and adjust for new Sfinae struct
Mikko Rasa [Fri, 24 May 2019 17:04:36 +0000 (20:04 +0300)]
Move NeedsCollection into its own header and adjust for new Sfinae struct

2 years agoUse automatic base detection when parsing integers
Mikko Rasa [Wed, 21 Feb 2018 17:30:40 +0000 (19:30 +0200)]
Use automatic base detection when parsing integers

3 years agoAdd a missing include
Mikko Rasa [Fri, 6 Jan 2017 02:54:59 +0000 (04:54 +0200)]
Add a missing include

4 years agoAdd a source class for builtin data
Mikko Rasa [Tue, 15 Nov 2016 15:57:21 +0000 (17:57 +0200)]
Add a source class for builtin data

4 years agoProvide access to the keyword of the current statement
Mikko Rasa [Sat, 5 Nov 2016 07:47:56 +0000 (09:47 +0200)]
Provide access to the keyword of the current statement

4 years agoReplace files from earlier packs with those from the newly added one
Mikko Rasa [Sat, 29 Oct 2016 16:45:40 +0000 (19:45 +0300)]
Replace files from earlier packs with those from the newly added one

4 years agoPrioritize matching suffixes when looking for a type
Mikko Rasa [Wed, 26 Oct 2016 21:12:46 +0000 (00:12 +0300)]
Prioritize matching suffixes when looking for a type

4 years agoMerge remote-tracking branch 'origin/master'
Mikko Rasa [Mon, 19 Sep 2016 14:00:06 +0000 (17:00 +0300)]
Merge remote-tracking branch 'origin/master'

4 years agoMove all exception classes to except.h, for real this time
Mikko Rasa [Mon, 19 Sep 2016 13:45:15 +0000 (16:45 +0300)]
Move all exception classes to except.h, for real this time

4 years agoImplement a parser mode for JSON files
Mikko Rasa [Fri, 2 Sep 2016 12:34:48 +0000 (15:34 +0300)]
Implement a parser mode for JSON files

4 years agoMove token-to-argument conversion to Statement
Mikko Rasa [Fri, 2 Sep 2016 12:34:15 +0000 (15:34 +0300)]
Move token-to-argument conversion to Statement

4 years agoSome refactoring of TextParser logic
Mikko Rasa [Fri, 2 Sep 2016 11:44:08 +0000 (14:44 +0300)]
Some refactoring of TextParser logic

4 years agoMove all exception classes to a common header
Mikko Rasa [Fri, 2 Sep 2016 11:41:49 +0000 (14:41 +0300)]
Move all exception classes to a common header

4 years agoDon't use deprecated typedefs
Mikko Rasa [Sun, 31 Jan 2016 15:37:38 +0000 (17:37 +0200)]
Don't use deprecated typedefs

4 years agoStore offsets and sizes in PackSource as 64-bit ints
Mikko Rasa [Mon, 7 Dec 2015 19:01:13 +0000 (21:01 +0200)]
Store offsets and sizes in PackSource as 64-bit ints

5 years agoAdd a getter for fallback in Collection
Mikko Rasa [Tue, 24 Nov 2015 20:19:06 +0000 (22:19 +0200)]
Add a getter for fallback in Collection

5 years agoUse const sources in Collection
Mikko Rasa [Wed, 18 Nov 2015 20:45:18 +0000 (22:45 +0200)]
Use const sources in Collection

The CollectionSource interface is entirely const anyway.

5 years agoChain get_status to fallback collection if one is defined
Mikko Rasa [Sat, 14 Nov 2015 17:05:03 +0000 (19:05 +0200)]
Chain get_status to fallback collection if one is defined

5 years agoRemove the loaded flag from PackSource files
Mikko Rasa [Fri, 13 Nov 2015 08:53:35 +0000 (10:53 +0200)]
Remove the loaded flag from PackSource files

It seems to be a relic from ancient times.  It also causes problems in
certain situations where it's desirable to remove and reload items.

5 years agoAdd an unpack option to the data tool
Mikko Rasa [Mon, 7 Sep 2015 09:26:58 +0000 (12:26 +0300)]
Add an unpack option to the data tool

5 years agoPlug a memory leak
Mikko Rasa [Sat, 7 Mar 2015 02:07:09 +0000 (04:07 +0200)]
Plug a memory leak

5 years agoAccept int-to-float conversion in array signature
Mikko Rasa [Fri, 19 Dec 2014 03:34:44 +0000 (05:34 +0200)]
Accept int-to-float conversion in array signature

6 years agoAdd version of add_pack_io with filter
Mikko Rasa [Mon, 10 Nov 2014 17:47:56 +0000 (19:47 +0200)]
Add version of add_pack_io with filter

6 years agoSupport adding packs from IO::Seekable objects
Mikko Rasa [Sun, 19 Oct 2014 17:33:05 +0000 (20:33 +0300)]
Support adding packs from IO::Seekable objects

This allows using any seekable objects as packs, such as Android assets.

6 years agoProvide help for datatool's command line options
Mikko Rasa [Sun, 19 Oct 2014 13:00:27 +0000 (16:00 +0300)]
Provide help for datatool's command line options

Also use add_argument instead of the deprecated get_args.

6 years agoMore resiliece against certain corner cases when direct loading
Mikko Rasa [Sun, 19 Oct 2014 08:58:00 +0000 (11:58 +0300)]
More resiliece against certain corner cases when direct loading

An empty binary file terminated with __end was causing problems as the
very first peek call would return null, causing Loader to fall back to
indirect loading.  However, the __end was already processed by peek, so
the parse call would fail.

6 years agoMake Loader a virtual base of ObjectLoader to permit diamond inheritance
Mikko Rasa [Wed, 15 Oct 2014 18:22:07 +0000 (21:22 +0300)]
Make Loader a virtual base of ObjectLoader to permit diamond inheritance

Useful when creating loader classes through composition.

6 years agoFix a class/struct inconsistency
Mikko Rasa [Sun, 5 Oct 2014 18:21:51 +0000 (21:21 +0300)]
Fix a class/struct inconsistency

6 years agoRequire sigc++-2.0 since PackSource uses it
Mikko Rasa [Sun, 5 Oct 2014 11:54:14 +0000 (14:54 +0300)]
Require sigc++-2.0 since PackSource uses it

The signal connection may be refactored into libmspcore in the future,
but it's required here for now.

6 years agoAdd a flag to control replacement of DirectorySource entries
Mikko Rasa [Sun, 5 Oct 2014 01:50:48 +0000 (04:50 +0300)]
Add a flag to control replacement of DirectorySource entries

6 years agoUse dynamic_cast to support Collection as virtual base class
Mikko Rasa [Sun, 20 Apr 2014 09:53:33 +0000 (12:53 +0300)]
Use dynamic_cast to support Collection as virtual base class

6 years agoFix logic problems when ignoring a statement in the middle of direct load
Mikko Rasa [Thu, 19 Dec 2013 21:58:53 +0000 (23:58 +0200)]
Fix logic problems when ignoring a statement in the middle of direct load

Each recursive call to parse() was subtracting one from sub_remaining,
causing it to get out of sync with the actual statement hierarchy.

6 years agoDetect eof from the parser, not input file
Mikko Rasa [Tue, 17 Dec 2013 01:01:06 +0000 (03:01 +0200)]
Detect eof from the parser, not input file

Some files have an embedded logical eof, so waiting for the end of the
physical file would (and did) cause an infinite loop.

7 years agoExpose raw open functionality from Collection
Mikko Rasa [Sat, 12 Oct 2013 17:10:45 +0000 (20:10 +0300)]
Expose raw open functionality from Collection

There are several scenarios where streaming or asynchronous loading is
useful.

7 years agoInvoke creators from get_list
Mikko Rasa [Sat, 12 Oct 2013 15:55:33 +0000 (18:55 +0300)]
Invoke creators from get_list

7 years agoAvoid duplicate names if an item is available from multiple sources
Mikko Rasa [Sat, 12 Oct 2013 15:21:30 +0000 (18:21 +0300)]
Avoid duplicate names if an item is available from multiple sources

7 years agoExport file metadata from PackSource
Mikko Rasa [Thu, 5 Sep 2013 14:53:35 +0000 (17:53 +0300)]
Export file metadata from PackSource

7 years agoDon't forget to initialize the control statement flag
Mikko Rasa [Mon, 19 Aug 2013 14:00:11 +0000 (17:00 +0300)]
Don't forget to initialize the control statement flag

7 years agoMore efficient way of loading binary files
Mikko Rasa [Mon, 5 Aug 2013 11:56:26 +0000 (14:56 +0300)]
More efficient way of loading binary files

The Variant array for storing arguments in Statement is too slow.  Bypass
it by passing args directly to the action in a sort of ad-hoc struct.

7 years agoMove newline eating out of BinaryParser
Mikko Rasa [Mon, 5 Aug 2013 09:32:43 +0000 (12:32 +0300)]
Move newline eating out of BinaryParser

It's really a feature of the mode change logic, not the binary mode
itself.

7 years agoRestructure control statement processing
Mikko Rasa [Mon, 5 Aug 2013 09:19:31 +0000 (12:19 +0300)]
Restructure control statement processing

Binary format control statements can no longer occur as substatements, but
that was an accidental feature at best.

7 years agoReplace local RAII set utility with one from mspcore
Mikko Rasa [Sat, 3 Aug 2013 12:24:01 +0000 (15:24 +0300)]
Replace local RAII set utility with one from mspcore

7 years agoUse a common StatementKey structure for Loader and BinaryParser/Writer
Mikko Rasa [Sat, 3 Aug 2013 12:14:11 +0000 (15:14 +0300)]
Use a common StatementKey structure for Loader and BinaryParser/Writer

7 years agoConsider full statement signature when dealing with auxiliary loaders
Mikko Rasa [Sun, 16 Jun 2013 08:25:58 +0000 (11:25 +0300)]
Consider full statement signature when dealing with auxiliary loaders

7 years agoAdd a way to combine multiple loaders
Mikko Rasa [Sat, 15 Jun 2013 20:22:23 +0000 (23:22 +0300)]
Add a way to combine multiple loaders

7 years agoAdd a two-argument constructor to DerivedObjectLoader
Mikko Rasa [Sat, 15 Jun 2013 20:10:12 +0000 (23:10 +0300)]
Add a two-argument constructor to DerivedObjectLoader

7 years agoUse correct keyword in forward declaration
Mikko Rasa [Fri, 24 May 2013 14:47:03 +0000 (17:47 +0300)]
Use correct keyword in forward declaration

clang isn't happy if a struct is forward declared as a class, even though
they are semantically equivalent in that context.

7 years agoAdd accessors for data_error members
Mikko Rasa [Fri, 24 May 2013 14:46:20 +0000 (17:46 +0300)]
Add accessors for data_error members

7 years agoAdd ability to put raw files in a pack
Mikko Rasa [Sat, 13 Apr 2013 12:57:11 +0000 (15:57 +0300)]
Add ability to put raw files in a pack

7 years agoMiscellaneous cleanups
Mikko Rasa [Tue, 26 Mar 2013 20:37:37 +0000 (22:37 +0200)]
Miscellaneous cleanups

7 years agoUse the Sfinae helper struct from mspcore
Mikko Rasa [Tue, 26 Mar 2013 20:37:08 +0000 (22:37 +0200)]
Use the Sfinae helper struct from mspcore

7 years agoComment improvements
Mikko Rasa [Tue, 26 Mar 2013 20:36:17 +0000 (22:36 +0200)]
Comment improvements

7 years agoReturn null instead of throwing if a file is not found in sources
Mikko Rasa [Tue, 26 Mar 2013 20:15:19 +0000 (22:15 +0200)]
Return null instead of throwing if a file is not found in sources

The primary use case for open_from_sources is for creator functions to
open raw files.  This change makes it easier to gracefully skip a non-
existent file and carry on with other ways of loading the item.

7 years agoAdd a virtual destructor to CollectionSource
Mikko Rasa [Wed, 16 Jan 2013 14:05:38 +0000 (16:05 +0200)]
Add a virtual destructor to CollectionSource

7 years agoAdd an API to open files from a collection's sources
Mikko Rasa [Wed, 16 Jan 2013 14:01:28 +0000 (16:01 +0200)]
Add an API to open files from a collection's sources

7 years agoUse IO::Slice to constrain access to logical files
Mikko Rasa [Sat, 5 Jan 2013 11:03:59 +0000 (13:03 +0200)]
Use IO::Slice to constrain access to logical files

7 years agoPass mode to ZlibCompressed
Mikko Rasa [Sat, 5 Jan 2013 09:40:42 +0000 (11:40 +0200)]
Pass mode to ZlibCompressed

This makes creating compressed packs work.  The temporary file is opened
with M_RDWR because we want to read the data back from the same handle, so
ZlibCompressed can't autodetect the mode.

7 years agoAdd a method to filter the files loaded from a pack
Mikko Rasa [Fri, 4 Jan 2013 17:29:58 +0000 (19:29 +0200)]
Add a method to filter the files loaded from a pack

7 years agoAdd support for fallback collections
Mikko Rasa [Fri, 7 Dec 2012 10:28:15 +0000 (12:28 +0200)]
Add support for fallback collections

7 years agoSome minor fixes in Collection
Mikko Rasa [Thu, 6 Dec 2012 21:44:11 +0000 (23:44 +0200)]
Some minor fixes in Collection

Move RemoveConst invovations close to the public interface to reduce the
number of template function instantiations.

Correct access specifiers.

7 years agoRework the list and containment queries
Mikko Rasa [Wed, 5 Dec 2012 19:16:00 +0000 (21:16 +0200)]
Rework the list and containment queries

Most of the implementations are now in .cpp, and they should deal better
with base classes.

7 years agoAdd some new test cases for Collection
Mikko Rasa [Wed, 5 Dec 2012 16:57:28 +0000 (18:57 +0200)]
Add some new test cases for Collection

Failures will be fixed a bit later.

8 years agoRestore the check to avoid deletion if an add call fails
Mikko Rasa [Tue, 4 Dec 2012 10:28:24 +0000 (12:28 +0200)]
Restore the check to avoid deletion if an add call fails

It was lost to a careless rewrite in b1bc256.

8 years agoUpdate Collection test cases with source and base class changes
Mikko Rasa [Tue, 4 Dec 2012 09:08:11 +0000 (11:08 +0200)]
Update Collection test cases with source and base class changes

8 years agoMore flexible system for handling base classes in Collection
Mikko Rasa [Tue, 4 Dec 2012 08:32:43 +0000 (10:32 +0200)]
More flexible system for handling base classes in Collection

Items are now stored with their original type, but can be retrieved with
a base class as well.

8 years agoTurn internal type helpers into structs
Mikko Rasa [Mon, 3 Dec 2012 21:58:06 +0000 (23:58 +0200)]
Turn internal type helpers into structs

Eliminating access specifiers and dummy constructors reduces clutter, and
in most cases pure virtual functions prevent instantiation anyway.  Since
these reside in private or protected sections of other classes, risk of
misuse is negligible.

8 years agoMove most of Collection::get implementation to collection.cpp
Mikko Rasa [Mon, 3 Dec 2012 21:20:02 +0000 (23:20 +0200)]
Move most of Collection::get implementation to collection.cpp

8 years agoUse an Fmt object to store the float format
Mikko Rasa [Sat, 1 Dec 2012 09:45:04 +0000 (11:45 +0200)]
Use an Fmt object to store the float format

8 years agoAdd the target type to lexical_cast invocations
Mikko Rasa [Sat, 1 Dec 2012 09:32:19 +0000 (11:32 +0200)]
Add the target type to lexical_cast invocations

8 years agoCache filenames in DirectorySource
Mikko Rasa [Thu, 15 Nov 2012 21:25:59 +0000 (23:25 +0200)]
Cache filenames in DirectorySource

8 years agoSimplify DirectorySource path management
Mikko Rasa [Thu, 15 Nov 2012 21:03:43 +0000 (23:03 +0200)]
Simplify DirectorySource path management

8 years agoRedesign automatic object loading
Mikko Rasa [Thu, 15 Nov 2012 19:04:09 +0000 (21:04 +0200)]
Redesign automatic object loading

Replace the Collection subclasses with a new concept of CollectionSources.
This permits more than one way of loading objects.

8 years agoFix comment handling
Mikko Rasa [Sun, 4 Nov 2012 19:15:40 +0000 (21:15 +0200)]
Fix comment handling

A comment start sequence inside another comment should not change the
comment type.

8 years agoAdd a test case for comments
Mikko Rasa [Sun, 4 Nov 2012 19:15:02 +0000 (21:15 +0200)]
Add a test case for comments

8 years agoAdd an intelligent packed collection class
Mikko Rasa [Fri, 28 Sep 2012 23:34:47 +0000 (02:34 +0300)]
Add an intelligent packed collection class

8 years agoThrow an exception if a future object couldn't be created
Mikko Rasa [Fri, 28 Sep 2012 23:01:34 +0000 (02:01 +0300)]
Throw an exception if a future object couldn't be created

8 years agoPreserve source declarations when transferring a collection
Mikko Rasa [Fri, 28 Sep 2012 22:55:03 +0000 (01:55 +0300)]
Preserve source declarations when transferring a collection

Optionally print all internal statements (useful to see what's happening
inside a binary datafile).

8 years agoAdd a flag to return also internal statements
Mikko Rasa [Fri, 28 Sep 2012 22:54:06 +0000 (01:54 +0300)]
Add a flag to return also internal statements

8 years agoAdd a special keyword to terminate parsing
Mikko Rasa [Fri, 28 Sep 2012 09:11:58 +0000 (12:11 +0300)]
Add a special keyword to terminate parsing

8 years agoImprove the API for future objects
Mikko Rasa [Fri, 28 Sep 2012 09:10:23 +0000 (12:10 +0300)]
Improve the API for future objects

add_future is now documented as ignoring names that don't match any type,
and there's a new function to add future objects with a keyword.

8 years agoRemove the outdated except.h
Mikko Rasa [Fri, 28 Sep 2012 09:04:38 +0000 (12:04 +0300)]
Remove the outdated except.h