Reads a SQLite database file directly, read only and with normal locking so that edits made by other programs show on the next read. More...

#include <sqlite_backend.h>

Inheritance diagram for SQLITE_BACKEND:

Public Member Functions
	SQLITE_BACKEND (std::string aPath, int aTimeoutSeconds)

	~SQLITE_BACKEND () override

bool	Connect () override

void	Disconnect () override

bool	IsConnected () const override

bool	ListTables (std::vector< std::string > &aTables) override
	Lists the tables and views, sorted by name.

std::optional< DB_TABLE_INFO >	DescribeTable (const std::string &aTable) override

bool	ReadTable (const DB_TABLE_INFO &aTable, const std::vector< std::string > &aColumns, size_t aMaxRows, std::vector< std::string > &aLabels, const DB_ROW_SINK &aSink) override
	Reads rows of a table, handing each to aSink.

std::string	GetLastError () const override

DB_FAILURE	GetLastFailure () const override

virtual bool	ListColumns (const std::string &aTable, std::vector< std::string > &aColumns)
	Lists the column names of a table as the catalog reports them, empty if it has none, false if the catalog failed.

virtual bool	ReadAllRows (const DB_TABLE_INFO &aTable, const std::vector< std::string > &aColumns, std::vector< std::string > &aLabels, const DB_ROW_SINK &aSink, const DB_RESTART_SINK &aRestart)
	Reads every row of a table for a caller that keeps them, handing each to aSink.

virtual void	BeginLoadCycle (const DB_STOP_CHECK &aStopped)
	Brackets a load cycle, in which the caller reads each table once.

virtual void	EndLoadCycle ()

virtual std::optional< bool >	ReadRow (const DB_TABLE_INFO &aTable, const std::vector< std::string > &aColumns, const std::string &aKeyColumn, const std::string &aKey, std::vector< std::string > &aLabels, std::vector< DB_VALUE > &aRow)
	Reads the one row whose key column equals aKey, using an indexed lookup if the backend has one.

Static Public Member Functions
static std::string	FileUri (const std::string &aPath, bool aWindowsPath)
	Builds the read-only SQLite URI for a file.

static bool	IsUncPath (const std::string &aPath)

static bool	IsNetworkPath (const std::string &aPath)

static bool	IsRemoteFuseMount (const std::string &aMountInfo, const std::string &aPath)
	Decides from the text of /proc/self/mountinfo whether aPath lies on a FUSE mount that reaches a remote machine.

static DB_BACKEND_RESULT	Create (const DATABASE_SOURCE &aSource, const wxString &aBasePath)
	Builds the backend for a source.

Static Public Attributes
static constexpr size_t	ALL_ROWS = std::numeric_limits<size_t>::max()
	Passed as ReadTable()'s row limit to read every row.

Private Member Functions
std::chrono::milliseconds	callerBound () const

template<typename R>
std::optional< R >	runOnWorker (std::function< R(DB_JOB_CONTEXT &)> aJob, const DB_ROW_SINK *aSink)
	Runs aJob on the worker, recording why it did not finish as the failure.

void	dropConnection ()
	Forgets the connection and closes it on a worker thread, never on the caller.

void	fail (DB_FAILURE aFailure, std::string aError)

void	failCall (DB_FAILURE aFailure, std::string aMessage)
	Records a failure of a call that ran, and drops the connection if the failure means it cannot be trusted.

bool	requireConnection ()
	Fails with a message and returns false unless a connection is open.

Private Attributes
std::string	m_path

int	m_timeout

std::unique_ptr< DB_WORKER_THREAD >	m_worker

std::shared_ptr< SQLITE_DB >	m_db

std::atomic< bool >	m_connected { false }

std::string	m_lastError

DB_FAILURE	m_lastFailure = DB_FAILURE::UNKNOWN

Detailed Description

Reads a SQLite database file directly, read only and with normal locking so that edits made by other programs show on the next read.

Every SQLite call runs on a worker thread, so a file on a share that stopped answering costs the caller a bounded wait, not a hung program.

Definition at line 43 of file sqlite_backend.h.

Constructor & Destructor Documentation

◆ SQLITE_BACKEND()

SQLITE_BACKEND::SQLITE_BACKEND	(	std::string	aPath,
		int	aTimeoutSeconds )

Parameters

aPath	the database file, as UTF-8
aTimeoutSeconds	how long SQLite waits on a lock; the caller never waits more than this plus 10 s

Definition at line 959 of file sqlite_backend.cpp.

References DB_WORKER_THREAD::CallerBound(), m_path, m_timeout, m_worker, and move.

◆ ~SQLITE_BACKEND()

SQLITE_BACKEND::~SQLITE_BACKEND ( )

override

Definition at line 968 of file sqlite_backend.cpp.

References Disconnect().

Member Function Documentation

◆ BeginLoadCycle()

virtual void DATABASE_BACKEND::BeginLoadCycle ( const DB_STOP_CHECK & aStopped )

inlinevirtualinherited

Brackets a load cycle, in which the caller reads each table once.

A backend may keep what it opens for the cycle's reads until the cycle ends. Cycles may nest, and only the outermost begin and end count. Nesting is backend-wide and may cross threads, as when a lookup on one thread reads a table while a refresh on another has a cycle open. The nested read keeps the stop check and sessions of the outermost cycle.

Parameters

aStopped may be empty; a backend that retries a read gives up when it returns true

Reimplemented in ACE_BACKEND.

Definition at line 161 of file database_backend.h.

◆ callerBound()

std::chrono::milliseconds SQLITE_BACKEND::callerBound ( ) const

private

Definition at line 953 of file sqlite_backend.cpp.

References DB_WORKER_THREAD::CallerBound(), and m_timeout.

Referenced by dropConnection(), and runOnWorker().

◆ Connect()

bool SQLITE_BACKEND::Connect ( )

overridevirtual

Implements DATABASE_BACKEND.

Definition at line 1086 of file sqlite_backend.cpp.

References _, describeFailure(), fail(), FILE_NOT_FOUND, FileUri(), m_connected, m_db, m_lastError, m_lastFailure, m_path, m_timeout, result, runOnWorker(), UNKNOWN, and utf8().

◆ Create()

DB_BACKEND_RESULT DATABASE_BACKEND::Create	(	const DATABASE_SOURCE &	aSource,
		const wxString &	aBasePath )

staticinherited

Builds the backend for a source.

Parameters

aSource	the source as configured; ${CWD} is expanded in its connection string and path
aBasePath	the directory ${CWD} stands for

Returns: the backend, or a null backend with the failure and a message naming the problem

Definition at line 34 of file database_backend.cpp.

References _, ACCESS, DATABASE_SOURCE::Expanded(), INVALID, INVALID_TYPE, ACE_BACKEND::MISSING, NOT_SUPPORTED_ON_PLATFORM, ODBC, PROVIDER_MISSING, ACE_BACKEND::ProviderMissingMessage(), ACE_BACKEND::ProviderState(), source, SQLITE, UNKNOWN, and utf8().

Referenced by SCH_IO_DATABASE::connectOnce(), and SCH_IO_DATABASE::CreateConnection().

◆ DescribeTable()

std::optional< DB_TABLE_INFO > SQLITE_BACKEND::DescribeTable ( const std::string & aTable )

overridevirtual

Returns: the table's columns in catalog order, or nullopt if it was not found or the catalog failed

Implements DATABASE_BACKEND.

Definition at line 1169 of file sqlite_backend.cpp.

References describeFailure(), failCall(), info, m_db, m_lastError, m_lastFailure, m_path, requireConnection(), result, runOnWorker(), and UNKNOWN.

◆ Disconnect()

void SQLITE_BACKEND::Disconnect ( )

overridevirtual

Implements DATABASE_BACKEND.

Definition at line 1130 of file sqlite_backend.cpp.

References dropConnection().

Referenced by ~SQLITE_BACKEND().

◆ dropConnection()

void SQLITE_BACKEND::dropConnection ( )

private

Forgets the connection and closes it on a worker thread, never on the caller.

Definition at line 1067 of file sqlite_backend.cpp.

References callerBound(), m_connected, m_db, and m_worker.

Referenced by Disconnect(), failCall(), ReadTable(), and runOnWorker().

◆ EndLoadCycle()

virtual void DATABASE_BACKEND::EndLoadCycle ( )

inlinevirtualinherited

Reimplemented in ACE_BACKEND.

Definition at line 163 of file database_backend.h.

◆ fail()

void SQLITE_BACKEND::fail	(	DB_FAILURE	aFailure,
		std::string	aError )

private

Definition at line 974 of file sqlite_backend.cpp.

References m_lastError, m_lastFailure, and traceDatabase.

Referenced by Connect(), failCall(), ReadTable(), requireConnection(), and runOnWorker().

◆ failCall()

void SQLITE_BACKEND::failCall	(	DB_FAILURE	aFailure,
		std::string	aMessage )

private

Records a failure of a call that ran, and drops the connection if the failure means it cannot be trusted.

Definition at line 1048 of file sqlite_backend.cpp.

References dropConnection(), fail(), and invalidates().

Referenced by DescribeTable(), and ListTables().

◆ FileUri()

std::string SQLITE_BACKEND::FileUri	(	const std::string &	aPath,
		bool	aWindowsPath )

static

Builds the read-only SQLite URI for a file.

Separators are made forward slashes and everything but unreserved characters is percent-encoded as UTF-8. A drive letter gives file:/C:/..., and a UNC path gives file:////server/share/..., since SQLite accepts only an empty authority.

Parameters

aWindowsPath true when backslashes separate directories, drive letters and UNC names apply

Definition at line 681 of file sqlite_backend.cpp.

References DB_WORKER_THREAD::NormalizePath(), and path.

Referenced by Connect().

◆ GetLastError()

std::string SQLITE_BACKEND::GetLastError ( ) const

inlineoverridevirtual

Implements DATABASE_BACKEND.

Definition at line 68 of file sqlite_backend.h.

References m_lastError.

◆ GetLastFailure()

DB_FAILURE SQLITE_BACKEND::GetLastFailure ( ) const

inlineoverridevirtual

Implements DATABASE_BACKEND.

Definition at line 70 of file sqlite_backend.h.

References m_lastFailure.

◆ IsConnected()

bool SQLITE_BACKEND::IsConnected ( ) const

inlineoverridevirtual

Implements DATABASE_BACKEND.

Definition at line 58 of file sqlite_backend.h.

References m_connected.

◆ IsNetworkPath()

bool SQLITE_BACKEND::IsNetworkPath ( const std::string & aPath )

static

Returns: true if the file lives on a network file system; false when that cannot be told

Definition at line 887 of file sqlite_backend.cpp.

References info, IsRemoteFuseMount(), IsUncPath(), L, and text.

Referenced by SQLITE_DB::Open().

◆ IsRemoteFuseMount()

bool SQLITE_BACKEND::IsRemoteFuseMount	(	const std::string &	aMountInfo,
		const std::string &	aPath )

static

Decides from the text of /proc/self/mountinfo whether aPath lies on a FUSE mount that reaches a remote machine.

Only the known remote subtypes (sshfs, rclone, s3fs, gcsfuse, smbnetfs, google-drive-ocamlfuse and gvfsd-fuse) count, since other FUSE file systems are local and keep their memory maps.

Definition at line 733 of file sqlite_backend.cpp.

References end, and result.

Referenced by IsNetworkPath().

◆ IsUncPath()

bool SQLITE_BACKEND::IsUncPath ( const std::string & aPath )

static

Returns: true for \server\share and \?\UNC\server\share paths, with either kind of slash

Definition at line 725 of file sqlite_backend.cpp.

References DB_WORKER_THREAD::NormalizePath(), and path.

Referenced by IsNetworkPath().

◆ ListColumns()

bool DATABASE_BACKEND::ListColumns	(	const std::string &	aTable,
		std::vector< std::string > &	aColumns )

virtualinherited

Lists the column names of a table as the catalog reports them, empty if it has none, false if the catalog failed.

Reimplemented in ODBC_BACKEND.

Definition at line 71 of file database_backend.cpp.

References DescribeTable(), GetLastError(), info, and DB_COLUMN::name.

◆ ListTables()

bool SQLITE_BACKEND::ListTables ( std::vector< std::string > & aTables )

overridevirtual

Lists the tables and views, sorted by name.

Implements DATABASE_BACKEND.

Definition at line 1136 of file sqlite_backend.cpp.

References describeFailure(), failCall(), m_db, m_lastError, m_path, requireConnection(), result, and runOnWorker().

◆ ReadAllRows()

virtual bool DATABASE_BACKEND::ReadAllRows	(	const DB_TABLE_INFO &	aTable,
		const std::vector< std::string > &	aColumns,
		std::vector< std::string > &	aLabels,
		const DB_ROW_SINK &	aSink,
		const DB_RESTART_SINK &	aRestart )

inlinevirtualinherited

Reads every row of a table for a caller that keeps them, handing each to aSink.

A backend that checks one read against another may deliver the table more than once, and calls aRestart before every delivery after the first, so the caller drops the rows it already took. The default reads once.

Parameters

aLabels is filled with the result-column labels before each delivery's first call to aSink

Returns: false on failure, in which case the caller discards any rows already delivered

Reimplemented in ACE_BACKEND.

Definition at line 146 of file database_backend.h.

References ALL_ROWS, and ReadTable().

◆ ReadRow()

virtual std::optional< bool > DATABASE_BACKEND::ReadRow	(	const DB_TABLE_INFO &	aTable,
		const std::vector< std::string > &	aColumns,
		const std::string &	aKeyColumn,
		const std::string &	aKey,
		std::vector< std::string > &	aLabels,
		std::vector< DB_VALUE > &	aRow )

inlinevirtualinherited

Reads the one row whose key column equals aKey, using an indexed lookup if the backend has one.

Returns: nullopt if the backend has no indexed lookup, otherwise true when aRow was filled

Reimplemented in ODBC_BACKEND.

Definition at line 170 of file database_backend.h.

◆ ReadTable()

bool SQLITE_BACKEND::ReadTable	(	const DB_TABLE_INFO &	aTable,
		const std::vector< std::string > &	aColumns,
		size_t	aMaxRows,
		std::vector< std::string > &	aLabels,
		const DB_ROW_SINK &	aSink )

overridevirtual

Reads rows of a table, handing each to aSink.

Parameters

aColumns	the columns to read; empty reads all of them
aMaxRows	stops after this many rows; zero fills aLabels and fetches no rows. A bounded read is ordered by the first column where the backend can.
aLabels	is filled with the result-column labels as the source spells them, before the first call to aSink

Returns: false on failure, in which case the caller discards any rows already delivered

Implements DATABASE_BACKEND.

Definition at line 1209 of file sqlite_backend.cpp.

References DATABASE_BACKEND::ALL_ROWS, describeFailure(), dropConnection(), DB_JOB_CONTEXT::Emit(), fail(), DB_READ_DELIVERY::Finish(), DB_JOB_CONTEXT::Flush(), invalidates(), DB_READ_DELIVERY::Labels(), m_db, m_lastError, m_path, DB_TABLE_INFO::name, requireConnection(), result, runOnWorker(), DB_READ_DELIVERY::Sink(), DB_READ_DELIVERY::SinkFailureMessage(), DB_READ_DELIVERY::SinkThrew(), and UNKNOWN.

◆ requireConnection()

bool SQLITE_BACKEND::requireConnection ( )

private

Fails with a message and returns false unless a connection is open.

Definition at line 1057 of file sqlite_backend.cpp.

References _, fail(), m_db, UNKNOWN, and utf8().

Referenced by DescribeTable(), ListTables(), and ReadTable().

◆ runOnWorker()

template<typename R>

std::optional< R > SQLITE_BACKEND::runOnWorker	(	std::function< R(DB_JOB_CONTEXT &)>	aJob,
		const DB_ROW_SINK *	aSink )

private

Runs aJob on the worker, recording why it did not finish as the failure.

Definition at line 1022 of file sqlite_backend.cpp.

References callerBound(), DB_WORKER_FAILURE::connection, DescribeWorkerOutcome(), DB_WORKER_FAILURE::DROP, dropConnection(), fail(), DB_WORKER_FAILURE::failure, DB_WORKER_FAILURE::FORGET, m_connected, m_db, m_path, m_worker, DB_WORKER_FAILURE::message, and result.

Referenced by Connect(), DescribeTable(), ListTables(), and ReadTable().

Member Data Documentation

◆ ALL_ROWS

size_t DATABASE_BACKEND::ALL_ROWS = std::numeric_limits<size_t>::max()

staticconstexprinherited

Passed as ReadTable()'s row limit to read every row.

Definition at line 106 of file database_backend.h.

Referenced by ReadAllRows(), ACE_BACKEND::ReadTable(), ODBC_BACKEND::ReadTable(), SQLITE_BACKEND::ReadTable(), and ACE_BACKEND::readVerified().

◆ m_connected

std::atomic<bool> SQLITE_BACKEND::m_connected { false }

private

Definition at line 118 of file sqlite_backend.h.

Referenced by Connect(), dropConnection(), IsConnected(), and runOnWorker().

◆ m_db

std::shared_ptr<SQLITE_DB> SQLITE_BACKEND::m_db

private

Definition at line 116 of file sqlite_backend.h.

Referenced by Connect(), DescribeTable(), dropConnection(), ListTables(), ReadTable(), requireConnection(), and runOnWorker().

◆ m_lastError

std::string SQLITE_BACKEND::m_lastError

private

Definition at line 120 of file sqlite_backend.h.

Referenced by Connect(), DescribeTable(), fail(), GetLastError(), ListTables(), and ReadTable().

◆ m_lastFailure

DB_FAILURE SQLITE_BACKEND::m_lastFailure = DB_FAILURE::UNKNOWN

private

Definition at line 121 of file sqlite_backend.h.

Referenced by Connect(), DescribeTable(), fail(), and GetLastFailure().

◆ m_path

std::string SQLITE_BACKEND::m_path

private

Definition at line 112 of file sqlite_backend.h.

Referenced by Connect(), DescribeTable(), ListTables(), ReadTable(), runOnWorker(), and SQLITE_BACKEND().

◆ m_timeout

int SQLITE_BACKEND::m_timeout

private

Definition at line 113 of file sqlite_backend.h.

Referenced by callerBound(), Connect(), and SQLITE_BACKEND().

◆ m_worker

std::unique_ptr<DB_WORKER_THREAD> SQLITE_BACKEND::m_worker

private

Definition at line 115 of file sqlite_backend.h.

Referenced by dropConnection(), runOnWorker(), and SQLITE_BACKEND().

The documentation for this class was generated from the following files:

Public Member Functions

Static Public Member Functions

Static Public Attributes

Private Member Functions

Private Attributes

Detailed Description

Constructor & Destructor Documentation

◆ SQLITE_BACKEND()

◆ ~SQLITE_BACKEND()

Member Function Documentation

◆ BeginLoadCycle()

◆ callerBound()

◆ Connect()

◆ Create()

◆ DescribeTable()

◆ Disconnect()

◆ dropConnection()

◆ EndLoadCycle()

◆ fail()

◆ failCall()

◆ FileUri()

◆ GetLastError()

◆ GetLastFailure()

◆ IsConnected()

◆ IsNetworkPath()

◆ IsRemoteFuseMount()

◆ IsUncPath()

◆ ListColumns()

◆ ListTables()

◆ ReadAllRows()

◆ ReadRow()

◆ ReadTable()

◆ requireConnection()

◆ runOnWorker()

Member Data Documentation

◆ ALL_ROWS

◆ m_connected

◆ m_db

◆ m_lastError

◆ m_lastFailure

◆ m_path

◆ m_timeout

◆ m_worker