E.163. Release 8.2

Allow the planner to reorder outer joins in some circumstances (Tom)

In previous releases, outer joins would always be evaluated in the order written in the query. This change allows the query optimizer to consider reordering outer joins, in cases where it can determine that the join order can be changed without altering the meaning of the query. This can make a considerable performance difference for queries involving multiple outer joins or mixed inner and outer joins.

Improve efficiency of IN (list-of-expressions) clauses (Tom)

Improve sorting speed and reduce memory usage (Simon, Tom)

Improve subtransaction performance (Alvaro, Itagaki Takahiro, Tom)

Add FILLFACTOR to table and index creation (ITAGAKI Takahiro)

This leaves extra free space in each table or index page, allowing improved performance as the database grows. This is particularly valuable to maintain clustering.

Increase default values for shared_buffers and max_fsm_pages (Andrew)

Improve locking performance by breaking the lock manager tables into sections (Tom)

This allows locking to be more fine-grained, reducing contention.

Reduce locking requirements of sequential scans (Qingqing Zhou)

Reduce locking required for database creation and destruction (Tom)

Improve the optimizer's selectivity estimates for LIKE, ILIKE, and regular expression operations (Tom)

Improve planning of joins to inherited tables and UNION ALL views (Tom)

Allow constraint exclusion to be applied to inherited UPDATE and DELETE queries (Tom)

SELECT already honored constraint exclusion.

Improve planning of constant WHERE clauses, such as a condition that depends only on variables inherited from an outer query level (Tom)

Protocol-level unnamed prepared statements are re-planned for each set of BIND values (Tom)

This improves performance because the exact parameter values can be used in the plan.

Speed up vacuuming of B-Tree indexes (Heikki Linnakangas, Tom)

Avoid extra scan of tables without indexes during VACUUM (Greg Stark)

Improve multicolumn GiST indexing (Oleg, Teodor)

Remove dead index entries before B-Tree page split (Junji Teramoto)

Allow a forced switch to a new transaction log file (Simon, Tom)

This is valuable for keeping warm standby slave servers in sync with the master. Transaction log file switching now also happens automatically during pg_stop_backup(). This ensures that all transaction log files needed for recovery can be archived immediately.

Add WAL informational functions (Simon)

Add functions for interrogating the current transaction log insertion point and determining WAL filenames from the hex WAL locations displayed by pg_stop_backup() and related functions.

Improve recovery from a crash during WAL replay (Simon)

The server now does periodic checkpoints during WAL recovery, so if there is a crash, future WAL recovery is shortened. This also eliminates the need for warm standby servers to replay the entire log since the base backup if they crash.

Improve reliability of long-term WAL replay (Heikki, Simon, Tom)

Formerly, trying to roll forward through more than 2 billion transactions would not work due to XID wraparound. This meant warm standby servers had to be reloaded from fresh base backups periodically.

Add archive_timeout to force transaction log file switches at a given interval (Simon)

This enforces a maximum replication delay for warm standby servers.

Add native LDAP authentication (Magnus Hagander)

This is particularly useful for platforms that do not support PAM, such as Windows.

Add GRANT CONNECT ON DATABASE (Gevik Babakhani)

This gives SQL-level control over database access. It works as an additional filter on top of the existing pg_hba.conf controls.

Add support for SSL Certificate Revocation List (CRL) files (Libor Hohoš)

The server and libpq both recognize CRL files now.

GiST indexes are now clusterable (Teodor)

Remove routine autovacuum server log entries (Bruce)

pg_stat_activity now shows autovacuum activity.

Track maximum XID age within individual tables, instead of whole databases (Alvaro)

This reduces the overhead involved in preventing transaction ID wraparound, by avoiding unnecessary VACUUMs.

Add last vacuum and analyze timestamp columns to the stats collector (Larry Rosenman)

These values now appear in the pg_stat_*_tables system views.

Improve performance of statistics monitoring, especially stats_command_string (Tom, Bruce)

This release enables stats_command_string by default, now that its overhead is minimal. This means pg_stat_activity will now show all active queries by default.

Add a waiting column to pg_stat_activity (Tom)

This allows pg_stat_activity to show all the information included in the ps display.

Add configuration parameter update_process_title to control whether the ps display is updated for every command (Bruce)

On platforms where it is expensive to update the ps display, it might be worthwhile to turn this off and rely solely on pg_stat_activity for status information.

Allow units to be specified in configuration settings (Peter)

For example, you can now set shared_buffers to 32MB rather than mentally converting sizes.

Add support for include directives in postgresql.conf (Joachim Wieland)

Improve logging of protocol-level prepare/bind/execute messages (Bruce, Tom)

Such logging now shows statement names, bind parameter values, and the text of the query being executed. Also, the query text is properly included in logged error messages when enabled by log_min_error_statement.

Prevent max_stack_depth from being set to unsafe values

On platforms where we can determine the actual kernel stack depth limit (which is most), make sure that the initial default value of max_stack_depth is safe, and reject attempts to set it to unsafely large values.

Enable highlighting of error location in query in more cases (Tom)

The server is now able to report a specific error location for some semantic errors (such as unrecognized column name), rather than just for basic syntax errors as before.

Fix "failed to re-find parent key" errors in VACUUM (Tom)

Clean out pg_internal.init cache files during server restart (Simon)

This avoids a hazard that the cache files might contain stale data after PITR recovery.

Fix race condition for truncation of a large relation across a gigabyte boundary by VACUUM (Tom)

Fix bug causing needless deadlock errors on row-level locks (Tom)

Fix bugs affecting multi-gigabyte hash indexes (Tom)

Each backend process is now its own process group leader (Tom)

This allows query cancel to abort subprocesses invoked from a backend or archive/recovery process.

Add INSERT/UPDATE/DELETE RETURNING (Jonah Harris, Tom)

This allows these commands to return values, such as the computed serial key for a new row. In the UPDATE case, values from the updated version of the row are returned.

Add support for multiple-row VALUES clauses, per SQL standard (Joe, Tom)

This allows INSERT to insert multiple rows of constants, or queries to generate result sets using constants. For example, INSERT ... VALUES (...), (...), ...., and SELECT * FROM (VALUES (...), (...), ....) AS alias(f1, ...).

Allow UPDATE and DELETE to use an alias for the target table (Atsushi Ogawa)

The SQL standard does not permit an alias in these commands, but many database systems allow one anyway for notational convenience.

Allow UPDATE to set multiple columns with a list of values (Susanne Ebrecht)

This is basically a short-hand for assigning the columns and values in pairs. The syntax is UPDATE tab SET (column, ...) = (val, ...).

Make row comparisons work per standard (Tom)

The forms <, <=, >, >= now compare rows lexicographically, that is, compare the first elements, if equal compare the second elements, and so on. Formerly they expanded to an AND condition across all the elements, which was neither standard nor very useful.

Add CASCADE option to TRUNCATE (Joachim Wieland)

This causes TRUNCATE to automatically include all tables that reference the specified table(s) via foreign keys. While convenient, this is a dangerous tool — use with caution!

Support FOR UPDATE and FOR SHARE in the same SELECT command (Tom)

Add IS NOT DISTINCT FROM (Pavel Stehule)

This operator is similar to equality (=), but evaluates to true when both left and right operands are NULL, and to false when just one is, rather than yielding NULL in these cases.

Improve the length output used by UNION/INTERSECT/EXCEPT (Tom)

When all corresponding columns are of the same defined length, that length is used for the result, rather than a generic length.

Allow ILIKE to work for multi-byte encodings (Tom)

Internally, ILIKE now calls lower() and then uses LIKE. Locale-specific regular expression patterns still do not work in these encodings.

Enable standard_conforming_strings to be turned on (Kevin Grittner)

This allows backslash escaping in strings to be disabled, making PostgreSQL more standards-compliant. The default is off for backwards compatibility, but future releases will default this to on.

Do not flatten subqueries that contain volatile functions in their target lists (Jaime Casanova)

This prevents surprising behavior due to multiple evaluation of a volatile function (such as random() or nextval()). It might cause performance degradation in the presence of functions that are unnecessarily marked as volatile.

Add system views pg_prepared_statements and pg_cursors to show prepared statements and open cursors (Joachim Wieland, Neil)

These are very useful in pooled connection setups.

Support portal parameters in EXPLAIN and EXECUTE (Tom)

This allows, for example, JDBC ? parameters to work in these commands.

If SQL-level PREPARE parameters are unspecified, infer their types from the content of the query (Neil)

Protocol-level PREPARE already did this.

Allow LIMIT and OFFSET to exceed two billion (Dhanaraj M)

Add TABLESPACE clause to CREATE TABLE AS (Neil)

This allows a tablespace to be specified for the new table.

Add ON COMMIT clause to CREATE TABLE AS (Neil)

This allows temporary tables to be truncated or dropped on transaction commit. The default behavior is for the table to remain until the session ends.

Add INCLUDING CONSTRAINTS to CREATE TABLE LIKE (Greg Stark)

This allows easy copying of CHECK constraints to a new table.

Allow the creation of placeholder (shell) types (Martijn van Oosterhout)

A shell type declaration creates a type name, without specifying any of the details of the type. Making a shell type is useful because it allows cleaner declaration of the type's input/output functions, which must exist before the type can be defined "for real". The syntax is CREATE TYPE typename.

Aggregate functions now support multiple input parameters (Sergey Koposov, Tom)

Add new aggregate creation syntax (Tom)

The new syntax is CREATE AGGREGATE aggname (input_type) (parameter_list). This more naturally supports the new multi-parameter aggregate functionality. The previous syntax is still supported.

Add ALTER ROLE PASSWORD NULL to remove a previously set role password (Peter)

Add DROP object IF EXISTS for many object types (Andrew)

This allows DROP operations on non-existent objects without generating an error.

Add DROP OWNED to drop all objects owned by a role (Alvaro)

Add REASSIGN OWNED to reassign ownership of all objects owned by a role (Alvaro)

This, and DROP OWNED above, facilitate dropping roles.

Add GRANT ON SEQUENCE syntax (Bruce)

This was added for setting sequence-specific permissions. GRANT ON TABLE for sequences is still supported for backward compatibility.

Add USAGE permission for sequences that allows only currval() and nextval(), not setval() (Bruce)

USAGE permission allows more fine-grained control over sequence access. Granting USAGE allows users to increment a sequence, but prevents them from setting the sequence to an arbitrary value using setval().

Add ALTER TABLE [ NO ] INHERIT (Greg Stark)

This allows inheritance to be adjusted dynamically, rather than just at table creation and destruction. This is very valuable when using inheritance to implement table partitioning.

Allow comments on global objects to be stored globally (Kris Jurka)

Previously, comments attached to databases were stored in individual databases, making them ineffective, and there was no provision at all for comments on roles or tablespaces. This change adds a new shared catalog pg_shdescription and stores comments on databases, roles, and tablespaces therein.

Add option to allow indexes to be created without blocking concurrent writes to the table (Greg Stark, Tom)

The new syntax is CREATE INDEX CONCURRENTLY. The default behavior is still to block table modification while an index is being created.

Provide advisory locking functionality (Abhijit Menon-Sen, Tom)

This is a new locking API designed to replace what used to be in /contrib/userlock. The userlock code is now on pgfoundry.

Allow COPY to dump a SELECT query (Zoltan Boszormenyi, Karel Zak)

This allows COPY to dump arbitrary SQL queries. The syntax is COPY (SELECT ...) TO.

Make the COPY command return a command tag that includes the number of rows copied (Volkan YAZICI)

Allow VACUUM to expire rows without being affected by other concurrent VACUUM operations (Hannu Krossing, Alvaro, Tom)

Make initdb detect the operating system locale and set the default DateStyle accordingly (Peter)

This makes it more likely that the installed postgresql.conf DateStyle value will be as desired.

Reduce number of progress messages displayed by initdb (Tom)

Allow full timezone names in timestamp input values (Joachim Wieland)

For example, '2006-05-24 21:11 America/New_York'::timestamptz.

Support configurable timezone abbreviations (Joachim Wieland)

A desired set of timezone abbreviations can be chosen via the configuration parameter timezone_abbreviations.

Add pg_timezone_abbrevs and pg_timezone_names views to show supported timezones (Magnus Hagander)

Add clock_timestamp(), statement_timestamp(), and transaction_timestamp() (Bruce)

clock_timestamp() is the current wall-clock time, statement_timestamp() is the time the current statement arrived at the server, and transaction_timestamp() is an alias for now().

Allow to_char() to print localized month and day names (Euler Taveira de Oliveira)

Allow to_char(time) and to_char(interval) to output AM/PM specifications (Bruce)

Intervals and times are treated as 24-hour periods, e.g. 25 hours is considered AM.

Add new function justify_interval() to adjust interval units (Mark Dilger)

Allow timezone offsets up to 14:59 away from GMT

Kiribati uses GMT+14, so we'd better accept that.

Interval computation improvements (Michael Glaesemann, Bruce)

Allow arrays to contain NULL elements (Tom)

Allow assignment to array elements not contiguous with the existing entries (Tom)

The intervening array positions will be filled with nulls. This is per SQL standard.

New built-in operators for array-subset comparisons (@>, <@, &&) (Teodor, Tom)

These operators can be indexed for many data types using GiST or GIN indexes.

Add convenient arithmetic operations on INET/CIDR values (Stephen R. van den Berg)

The new operators are & (and), | (or), ~ (not), inet + int8, inet - int8, and inet - inet.

Add new aggregate functions from SQL:2003 (Neil)

The new functions are var_pop(), var_samp(), stddev_pop(), and stddev_samp(). var_samp() and stddev_samp() are merely renamings of the existing aggregates variance() and stddev(). The latter names remain available for backward compatibility.

Add SQL:2003 statistical aggregates (Sergey Koposov)

New functions: regr_intercept(), regr_slope(), regr_r2(), corr(), covar_samp(), covar_pop(), regr_avgx(), regr_avgy(), regr_sxy(), regr_sxx(), regr_syy(), regr_count().

Allow domains to be based on other domains (Tom)

Properly enforce domain CHECK constraints everywhere (Neil, Tom)

For example, the result of a user-defined function that is declared to return a domain type is now checked against the domain's constraints. This closes a significant hole in the domain implementation.

Fix problems with dumping renamed SERIAL columns (Tom)

The fix is to dump a SERIAL column by explicitly specifying its DEFAULT and sequence elements, and reconstructing the SERIAL column on reload using a new ALTER SEQUENCE OWNED BY command. This also allows dropping a SERIAL column specification.

Add a server-side sleep function pg_sleep() (Joachim Wieland)

Add all comparison operators for the tid (tuple id) data type (Mark Kirkwood, Greg Stark, Tom)

Add TG_table_name and TG_table_schema to trigger parameters (Andrew)

TG_relname is now deprecated. Comparable changes have been made in the trigger parameters for the other PLs as well.

Allow FOR statements to return values to scalars as well as records and row types (Pavel Stehule)

Add a BY clause to the FOR loop, to control the iteration increment (Jaime Casanova)

Add STRICT to SELECT INTO (Matt Miller)

STRICT mode throws an exception if more or less than one row is returned by the SELECT, for Oracle PL/SQL compatibility.

Add table_name and table_schema to trigger parameters (Adam Sjøgren)

Add prepared queries (Dmitry Karasik)

Make $_TD trigger data a global variable (Andrew)

Previously, it was lexical, which caused unexpected sharing violations.

Run PL/Perl and PL/PerlU in separate interpreters, for security reasons (Andrew)

In consequence, they can no longer share data nor loaded modules. Also, if Perl has not been compiled with the requisite flags to allow multiple interpreters, only one of these languages can be used in any given backend process.

Named parameters are passed as ordinary variables, as well as in the args[] array (Sven Suursoho)

Add table_name and table_schema to trigger parameters (Andrew)

Allow returning of composite types and result sets (Sven Suursoho)

Return result-set as list, iterator, or generator (Sven Suursoho)

Allow functions to return void (Neil)

Python 2.5 is now supported (Tom)

Add new command \password for changing role password with client-side password encryption (Peter)

Allow \c to connect to a new host and port number (David, Volkan YAZICI)

Add tablespace display to \l+ (Philip Yarra)

Improve \df slash command to include the argument names and modes (OUT or INOUT) of the function (David Fetter)

Support binary COPY (Andreas Pflug)

Add option to run the entire session in a single transaction (Simon)

Use option -1 or --single-transaction.

Support for automatically retrieving SELECT results in batches using a cursor (Chris Mair)

This is enabled using \set FETCH_COUNT n. This feature allows large result sets to be retrieved in psql without attempting to buffer the entire result set in memory.

Make multi-line values align in the proper column (Martijn van Oosterhout)

Field values containing newlines are now displayed in a more readable fashion.

Save multi-line statements as a single entry, rather than one line at a time (Sergey E. Koposov)

This makes up-arrow recall of queries easier. (This is not available on Windows, because that platform uses the native command-line editing present in the operating system.)

Make the line counter 64-bit so it can handle files with more than two billion lines (David Fetter)

Report both the returned data and the command status tag for INSERT/UPDATE/DELETE RETURNING (Tom)

Allow complex selection of objects to be included or excluded by pg_dump (Greg Sabino Mullane)

pg_dump now supports multiple -n (schema) and -t (table) options, and adds -N and -T options to exclude objects. Also, the arguments of these switches can now be wild-card expressions rather than single object names, for example -t 'foo*', and a schema can be part of a -t or -T switch, for example -t schema1.table1.

Add pg_restore --no-data-for-failed-tables option to suppress loading data if table creation failed (i.e., the table already exists) (Martin Pitt)

Add pg_restore option to run the entire session in a single transaction (Simon)

Use option -1 or --single-transaction.

Add PQencryptPassword() to encrypt passwords (Tom)

This allows passwords to be sent pre-encrypted for commands like ALTER ROLE ... PASSWORD.

Add function PQisthreadsafe() (Bruce)

This allows applications to query the thread-safety status of the library.

Add PQdescribePrepared(), PQdescribePortal(), and related functions to return information about previously prepared statements and open cursors (Volkan YAZICI)

Allow LDAP lookups from pg_service.conf (Laurenz Albe)

Allow a hostname in ~/.pgpass to match the default socket directory (Bruce)

A blank hostname continues to match any Unix-socket connection, but this addition allows entries that are specific to one of several postmasters on the machine.

Allow SHOW to put its result into a variable (Joachim Wieland)

Add COPY TO STDOUT (Joachim Wieland)

Add regression tests (Joachim Wieland, Michael)

Major source code cleanups (Joachim Wieland, Michael)

Allow MSVC to compile the PostgreSQL server (Magnus, Hiroshi Saito)

Add MSVC support for utility commands and pg_dump (Hiroshi Saito)

Add support for Windows code pages 1253, 1254, 1255, and 1257 (Kris Jurka)

Drop privileges on startup, so that the server can be started from an administrative account (Magnus)

Stability fixes (Qingqing Zhou, Magnus)

Add native semaphore implementation (Qingqing Zhou)

The previous code mimicked SysV semaphores.

Add GIN (Generalized Inverted iNdex) index access method (Teodor, Oleg)

Remove R-tree indexing (Tom)

Rtree has been re-implemented using GiST. Among other differences, this means that rtree indexes now have support for crash recovery via write-ahead logging (WAL).

Reduce libraries needlessly linked into the backend (Martijn van Oosterhout, Tom)

Add a configure flag to allow libedit to be preferred over GNU readline (Bruce)

Use configure --with-libedit-preferred.

Allow installation into directories containing spaces (Peter)

Improve ability to relocate installation directories (Tom)

Add support for Solaris x86_64 using the Solaris compiler (Pierre Girard, Theo Schlossnagle, Bruce)

Add DTrace support (Robert Lor)

Add PG_VERSION_NUM for use by third-party applications wanting to test the backend version in C using > and < comparisons (Bruce)

Add XLOG_BLCKSZ as independent from BLCKSZ (Mark Wong)

Add LWLOCK_STATS define to report locking activity (Tom)

Emit warnings for unknown configure options (Martijn van Oosterhout)

Add server support for "plugin" libraries that can be used for add-on tasks such as debugging and performance measurement (Korry Douglas)

This consists of two features: a table of "rendezvous variables" that allows separately-loaded shared libraries to communicate, and a new configuration parameter local_preload_libraries that allows libraries to be loaded into specific sessions without explicit cooperation from the client application. This allows external add-ons to implement features such as a PL/pgSQL debugger.

Rename existing configuration parameter preload_libraries to shared_preload_libraries (Tom)

This was done for clarity in comparison to local_preload_libraries.

Add new configuration parameter server_version_num (Greg Sabino Mullane)

This is like server_version, but is an integer, e.g. 80200. This allows applications to make version checks more easily.

Add a configuration parameter seq_page_cost (Tom)

Re-implement the regression test script as a C program (Magnus, Tom)

Allow loadable modules to allocate shared memory and lightweight locks (Marc Munro)

Add automatic initialization and finalization of dynamically loaded libraries (Ralf Engelschall, Tom)

New functions _PG_init() and _PG_fini() are called if the library defines such symbols. Hence we no longer need to specify an initialization function in shared_preload_libraries; we can assume that the library used the _PG_init() convention instead.

Add PG_MODULE_MAGIC header block to all shared object files (Martijn van Oosterhout)

The magic block prevents version mismatches between loadable object files and servers.

Add shared library support for AIX (Laurenz Albe)

New XML documentation section (Bruce)

Major tsearch2 improvements (Oleg, Teodor)

• multibyte encoding support, including UTF8

• query rewriting support

• improved ranking functions

• thesaurus dictionary support

• Ispell dictionaries now recognize MySpell format, used by OpenOffice

• GIN support

Add adminpack module containing Pgadmin administration functions (Dave)

These functions provide additional file system access routines not present in the default PostgreSQL server.

Add sslinfo module (Victor Wagner)

Reports information about the current connection's SSL certificate.

Add pgrowlocks module (Tatsuo)

This shows row locking information for a specified table.

Add hstore module (Oleg, Teodor)

Add isn module, replacing isbn_issn (Jeremy Kronuz)

This new implementation supports EAN13, UPC, ISBN (books), ISMN (music), and ISSN (serials).

Add index information functions to pgstattuple (ITAGAKI Takahiro, Satoshi Nagayasu)

Add pg_freespacemap module to display free space map information (Mark Kirkwood)

pgcrypto now has all planned functionality (Marko Kreen)

• Include iMath library in pgcrypto to have the public-key encryption functions always available.

• Add SHA224 algorithm that was missing in OpenBSD code.

• Activate builtin code for SHA224/256/384/512 hashes on older OpenSSL to have those algorithms always available.

• New function gen_random_bytes() that returns cryptographically strong randomness. Useful for generating encryption keys.

• Remove digest_exists(), hmac_exists() and cipher_exists() functions.

Improvements to cube module (Joshua Reich)

New functions are cube(float[]), cube(float[], float[]), and cube_subset(cube, int4[]).

Add async query capability to dblink (Kai Londenberg, Joe Conway)

New operators for array-subset comparisons (@>, <@, &&) (Tom)

Various contrib packages already had these operators for their datatypes, but the naming wasn't consistent. We have now added consistently named array-subset comparison operators to the core code and all the contrib packages that have such functionality. (The old names remain available, but are deprecated.)

Add uninstall scripts for all contrib packages that have install scripts (David, Josh Drake)