Work was done to clean up the source code base, including: Removing unneeded CMake checks; removing unused macros from source files; reorganizing header files to reduce the number of dependencies and make them more modular, removing function declarations without definitions, replacing locally written functions with equivalent functions from industry-standard libraries.

In MySQL 5.7.6, the PASSWORD() function was deprecated, but no warning was produced when it was invoked. Similarly, the old_passwords system variable was deprecated, but no warning was produced when it was set. (Bug #20545464)

The deprecated IDENTIFIED BY PASSWORD syntax is supported only for CREATE USER and GRANT, but ALTER USER failed to reject it. ALTER USER now produces a syntax error. (Bug #76048, Bug #20600865)

SET PASSWORD ... = PASSWORD('auth_string') syntax was to be deprecated in MySQL 5.7.6, but was made illegal. This syntax is now available again, but generates a warning due to its deprecated status. These alternatives remain available, the first of which now should be considered the preferred form:

ALTER USER ... IDENTIFIED BY 'auth_string';
SET PASSWORD ... = 'auth_string';

(Bug #75927, Bug #20552143)

In MySQL 5.7.6, the NO_AUTO_CREATE_USER was deprecated. (It is preferable to create MySQL accounts with CREATE USER rather than GRANT.) Now the default SQL mode includes NO_AUTO_CREATE_USER and assignments to sql_mode that change the NO_AUTO_CREATE_USER mode state produce a warning, except assignments that set sql_mode to DEFAULT. NO_AUTO_CREATE_USER will be removed in a future MySQL version, at which point its effect will be enabled at all times (GRANT will not create accounts). (WL #8326)

Installers for more binary distribution types provide secure deployment. This includes installers for SLES, Solaris, OS X, FreeBSD, and Linux generic binary compressed tar distributions. These installers create a single 'root'@'localhost' account without other root or anonymous-user accounts, and do not create a test database accessible by any user. (Some installers may provide options to create additional accounts or a test database, but only if selected by the user.) Installers that run interactively and can ask the installing user for the initial root password do so. Noninteractive installers generate a random root password that the administrator can use to connect to the server the first time and choose a new password. (WL #7787)

It is now possible to provide hints to the optimizer within individual SQL statements, which enables finer control over statement execution plans than can be achieved using the optimizer_switch system variable. Optimizer hints are specified as /*+ ... */ comments following the SELECT, INSERT, REPLACE, UPDATE, or DELETE keyword of statements or query blocks. Hints are also permitted in statements used with EXPLAIN, enabling you to see how hints affect execution plans. Examples:

SELECT /*+ NO_RANGE_OPTIMIZATION(t3 PRIMARY, f2_idx) */ f1
  FROM t3 WHERE f1 > 30 AND f1 < 33;
SELECT /*+ BKA(t1) NO_BKA(t2) */ * FROM t1 INNER JOIN t2 WHERE ...;
SELECT /*+ NO_ICP(t1, t2) */ * FROM t1 INNER JOIN t2 WHERE ...;
EXPLAIN SELECT /*+ NO_ICP(t1) */ * FROM t1 WHERE ...;

For more information, see Optimizer Hints. (WL #8016, WL #8017, WL #8241, WL #8243)

Several binary distribution types have been made more modular, to split out test components into a separate distribution file. This reduces the size of the main download. In addition to the previously available test/debug distributions already available for Windows Zip archives, RPM packages, and Debian packages, the current release makes separate test distributions available for Solaris PKG files, and generic binary Linux and OS X compressed tar packages. These separate distributions have “test” in the distribution file name.

Generally, use of a test distribution requires that the main distribution is also installed. Additionally, for Solaris, the main and test distributions must be for the same version of MySQL. (WL #7787)

References: See also: Bug #20613327, Bug #20546298.

The events_statements_history and events_transactions_history consumers now are enabled by default. (WL #8321)

References: See also: Bug #71207, Bug #18376132.

Previously, proxy user mapping was available only for authentication plugins that implemented that capability for themselves. The MySQL server itself now can map proxy users according to granted proxy privileges. If the new check_proxy_users system variable is enabled, the server performs proxy user mapping for any authentication plugin that requests it. By default, check_proxy_users is disabled, so the server performs no proxy user mapping even for authentication plugins that request it.

In addition, the mysql_native_password and sha256_password built-in authentication plugins have been modified to take advantage of this server capability, and thus now are able to support proxy users. The new mysql_native_password_proxy_users and sha256_password_proxy_users system variables control whether each plugin requests proxy user mapping. By default, both variables are disabled, which produces behavior that is backward compatible with previous releases.

For information about user proxying, see Proxy Users. (WL #7724)

The C client library now attempts to establish an encrypted connection by default if the server supports encrypted connections. This affects client programs as follows:

For more information, see Command Options for Encrypted Connections.

This change affects these standard MySQL client programs: mysql, mysql_config_editor, mysql_install_db, mysql_plugin, mysql_secure_installation, mysql_upgrade, mysqladmin, mysqlbinlog, mysqlcheck, mysqldump, mysqlimport, mysqlshow, and mysqlslap. It will also affect new releases of MySQL Connectors that are based on the C client library: Connector/C, Connector/C++, and Connector/ODBC. (WL #7712)

The ST_Buffer(), ST_Difference(), ST_Distance(), ST_Intersection(), ST_IsSimple(), ST_SymDifference(), and ST_Union() functions have been reimplemented to use the functionality available in Boost.Geometry. The functions may raise an exception for invalid geometry argument values when the previous implementation may not have.

In addition, ST_Buffer() now takes up to three optional arguments to specify point, join, and end strategies that influence buffer computation. Values for strategy arguments are produced using the new ST_Buffer_Strategy() function. See Spatial Operator Functions. (WL #7225, WL #7929)

MySQL distributions now include the sys schema, a set of objects that helps DBAs and developers interpret data collected by the Performance Schema. sys schema objects can be used for typical tuning and diagnosis use cases.

For new installations, the sys schema is installed by default during data directory initialization if you use mysqld with the --initialize or --initialize-insecure option, or if you use mysql_install_db. To permit this behavior to be suppressed, mysql_install_db now has a --skip-sys-schema option. mysqld has no such option, but if you initialize the data directory using mysqld --initialize (or --initialize-insecure) rather than mysql_install_db, you can drop the sys schema manually after initialization if it is unneeded.

For upgrades, mysql_upgrade installs the sys schema if it is not installed, and upgrades it to the current version otherwise. To permit this behavior to be suppressed, mysql_upgrade now has a --skip-sys-schema option.

mysql_upgrade returns an error if a sys schema exists but has no version view, on the assumption that absence of this view indicates a user-created sys schema. To upgrade in this case, remove or rename the existing sys schema first.

If you upgrade to this MySQL release from an earlier version, you must run mysql_upgrade to install the sys schema.

For more information, see MySQL sys Schema. (WL #8159)

Important Change; InnoDB: The following changes were made to InnoDB configuration option default values:

The configuration parameter default changes described above may affect replication and mysqldump operations. Consider the following recommendations when using the new default settings:

The following file format related configuration options are deprecated and will be removed in a future MySQL version:

These four configuration options were provided for creating tables compatible with earlier versions of InnoDB in MySQL 5.1. Now that MySQL 5.1 has reached the end of its product lifecycle, these options are no longer required. Also, the file format scheme, by which named file formats would be introduced as new features were added, was not used after introduction of the Barracuda file format. InnoDB formats have changed since the introduction of the Barracuda file format, but new named file formats have not been added.

If non-default values are used for any of the four deprecated options, InnoDB prints a deprecation and removal warning to the server error log. The same warning is issued to the client if the parameters are set dynamically using a SET statement. (WL #7703)

Important Change; InnoDB: The innodb_buffer_pool_dump_at_shutdown and innodb_buffer_pool_load_at_startup configuration options are now enabled by default. With this change, a percentage of most-recently-used buffer pool pages is dumped at server shutdown and restored at server startup. This behavior helps avoid a lengthy buffer pool warmup period after restarting the server, particularly for instances with large buffer pools. The innodb_buffer_pool_dump_pct option defines the percentage of buffer pool pages that are dumped. The default value for innodb_buffer_pool_dump_pct is reduced from 100 to 25.

These configuration option default value changes represent a change in behaviour at server shutdown and startup. If you prefer the previous default values, it is recommended that you configure the options explicitly in your MySQL configuration file after upgrading to MySQL 5.7.7 or later, and before restarting the server.

For more information, see Saving and Restoring the Buffer Pool State. (WL #8317)

Important Change; InnoDB: The innodb_checksum_algorithm default value is now crc32. The previous default setting was innodb. This change also means that innodb_checksums=ON is now equivalent to innodb_checksum_algorithm=crc32 instead of innodb_checksum_algorithm=innodb. (WL #8315)

InnoDB: The InnoDB MeCab full-text parser plugin now supports the eucjpms, cp932, and utf8mb4 character sets. (Bug #20534096)

InnoDB: To address a scalability bottleneck for some workloads where LOCK_grant is locked in read-mode, LOCK_grant locks are now partitioned. Read lock requests on LOCK_grant now acquire one of multiple LOCK_grant partitions. Write locks must acquire all partitions.

To address another scalability bottleneck, the server no longer performs unnecessary lock acquisitions when creating interal temporary tables. (WL #8355, WL #8356)

References: See also: Bug #72829, Bug #20023139.

Replication: The defaults of some replication related variables have been modified. The following changes have been made:

Additionally, the session scope of gtid_executed has been deprecated. The global scope of gtid_executed remains supported. (WL #8313, WL #8314, WL #8319, WL #8320, WL #7518)

Replication: The XA implementation in MySQL has been made much more compatible with the XA specification. A prepared XA transaction is no longer rolled back at disconnect. Now when replicating using the binary log, an XA transaction in PREPARED state persists in the binary log until an explicit XA COMMIT or XA ROLLBACK statement is issued. In prior versions, an XA transaction that was in PREPARED state would be rolled back on clean server shutdown or client disconnect. Similarly, an XA transaction that was in PREPARED state would still exist in PREPARED state in case the server was shut down abnormally and then started again, but the contents of the transaction could not be written to the binary log. As part of this feature, a new event, XA_prepare_log_event, has been added to track XA transactions in the PREPARED state and enable them to be replicated. To finalize a two-phase XA transaction, the XA COMMIT or XA ROLLBACK is recorded separately in the binary log, possibly interleaving with other transactions. XA transactions committed with the new XA COMMIT ONE PHASE syntax are logged as one part using XA_prepare_log_event. (WL #6860)

References: See also: Bug #12161, Bug #11745231, Bug #75204, Bug #20214365.

Performance Schema stage event instruments that provide statement progress information now are enabled and timed by default. The affected instruments are those displayed by this statement:

SELECT * FROM performance_schema.setup_instruments WHERE
ENABLED='YES' AND NAME LIKE "stage/%";

(Bug #20685859)

The XID column of Performance Schema transaction tables (for example, events_transactions_current) has been split into three columns to permit access to the component parts of XID values. The new columns are XID_FORMAT_ID, XID_GTRID, and XID_BQUAL. (Bug #18320361)

Previously, view definitions were not permitted to contain derived tables (subqueries) in the FROM clause. This restriction has now been lifted.

Previously, derived tables (subqueries) or views in the FROM clause could not be merged into the outer query if they contained subqueries in the SELECT list. Instead, they were processed using materialization. This restriction has now been lifted so that queries previously executed using materialization can be executed more quickly using merging. (Bug #12755, Bug #11745276, Bug #60417, Bug #11865600)

InnoDB: FLUSH TABLES ... FOR EXPORT, which is an unsupported operation for tables residing in a general tablespace, failed to report a warning message. (Bug #20631305)

InnoDB: The MeCab parser accessed the byte beyond the length of the document resulting in an Invalid read of size 1 Valgrind error. Document allocation size is now length + 1 byte. (Bug #20589432)

InnoDB: A debug assertion should not be raised for static mutexes. (Bug #20588765)

InnoDB: Tablespace and file path data updates to internal system tables during startup caused undo log corruption. The updates were made before calling truncate::fixup_tables(), which does not expect to encounter changed pages. (Bug #20534616)

InnoDB: A TRUNCATE TABLE operation on a temporary table raised an assertion. The temporary table object was incompletely constructed when reloaded from SYS_TABLES. (Bug #20527363, Bug #72080)

InnoDB: Return value 16 when calling pthread_mutex_destroy() warnings were returned during atexit() processing. (Bug #20513522)

InnoDB: A debug variable used to modify the first page of a tablespace raised an assertion when set to a nonexistent tablespace ID. (Bug #20511314, Bug #75833, Bug #19865274, Bug #74481)

InnoDB: A buffer pool load operation raised an assertion when attempting to read pages that were placed out of tablespace bounds by a preceding TRUNCATE TABLE operation. (Bug #20474542)

InnoDB: Removal of a foreign key object from the data dictionary cache during error handling caused the server to exit. (Bug #20442523)

InnoDB: An assertion was raised during a redo log resize operation that was triggered by a file size mismatch encountered during recovery. Code introduced in MySQL 5.6.8 to automatically handle redo log file size mismatches failed to ensure that the buffer pool was clean prior to recreating redo log files. (Bug #20425387)

InnoDB: An InnoDB memcached extra_col_value[] array was freed without checking the allocated flag, causing a server exit. (Bug #20400373)

InnoDB: A DML operation performed while a flushing operation was in progress raised a memcached-related assertion. (Bug #20390277)

InnoDB: A CHECK TABLE operation on a table with a spatial index raised an assertion. The row_sel_sec_rec_is_for_clust_rec function failed to handle an externally stored field. (Bug #20311344)

InnoDB: CHECK TABLE reported misplaced rows after an in-place ALTER TABLE operation on a “partitioned by key” table. An in-place operation that drops and adds the primary key should not be permitted for tables that are partitioned by key. (Bug #20190520)

InnoDB: Estimates that were too low for the size of merge chunks in the result sorting algorithm caused a server exit. (Bug #20049521)

InnoDB: An ALTER TABLE ... RENAME operation raised an invalid assertion. The assertion code used an incorrect transaction object. (Bug #18523599)

References: This issue is a regression of: Bug #17447500.

InnoDB: The system tablespace and undo tablepaces were missing from INNODB_SYS_DATAFILES and INNODB_SYS_TABLESPACES output. (Bug #16904899, Bug #69323)

InnoDB: Running mysql_upgrade after a binary upgrade to MySQL 5.7.6 caused a server exit on a system with tablespace data files that were created in MySQL 5.1 or earlier. The fix for Bug #17345513 in MySQL 5.7.6 failed to address all instances of garbage FIL_PAGE_TYPE values in tablespace data files created in MySQL 5.1 or earlier.

With this patch, the manual process described in the MySQL 5.7.6 release notes entry for Bug #17345513 for repairing non-index pages that contain invalid FIL_PAGE_TYPE values is no longer necessary.

The patch for Bug #17345513 also failed to recompute page checksums after resetting invalid FIL_PAGE_TYPE values. Upon restarting the server, a failure would occur due to an apparent page corruption. The page checksum is now recomputed before the new FIL_PAGE_TYPE value is written to the data file. (Bug #76262, Bug #20691930)

References: This issue is a regression of: Bug #17345513.

InnoDB: For full-text searches, the optimizer could choose an index that does not produce correct relevancy rankings. (Bug #74686, Bug #19950568)

Partitioning: The MySQL Server unnecessarily requested the default number of partitions for a table whenever it opened a partitioned table. This was unnecessary since the server already has this information about the table to be opened. Now the server requests this information only when needed—that is, only if it is creating or altering a partitioned table. (Bug #76007, Bug #20585753)

Partitioning: A number of ALTER TABLE statements that attempted to add partitions, columns, or indexes to a partitioned table while a write lock was in effect for this table were not handled correctly. (Bug #74451, Bug #74478, Bug #74491, Bug #74560, Bug #74746, Bug #74841, Bug #74860, Bug #74869, Bug #19856162, Bug #19864284, Bug #19873019, Bug #19891663, Bug #19990815, Bug #20026661, Bug #20031966, Bug #20033503, Bug #19827845)

Partitioning: Executing an ALTER TABLE on a partitioned table on which a write lock was in effect could cause subsequent SQL statements on this table to fail. (Bug #74288, Bug #74634, Bug #19784790, Bug #19918805)

References: See also: Bug #19856162, Bug #74451.

Replication: Some memory copy operations being performed on the replication_connection_status Performance Schema table were using an incorrect length, which could lead to a buffer overflow error or truncated output. The fix ensures that the correct length is used. (Bug #20535692)

Replication: When using multi-source replication on a multithreaded slave (where slave_parallel_workers is greater than 1), and slave_transaction_retries was greater than 1, the slave would fail to open the relay log file. This was due to the slave worker incorrectly constructing the relay log file path for its replication channel. (Bug #20448413)

Replication: When the automatic_sp_privileges variable is set, the server automatically grants the EXECUTE and ALTER ROUTINE privileges to the creator of a stored routine, if the user does not already have these privileges. When a privileged user creates a procedure with DEFINER as a non privileged user on a master, the current user is considered to be a privileged user and the mysql.procs_priv table is not updated. When such a statement was replicated to slave, the non-privileged DEFINER was considered as the current user on the slave and privileges were being allocated. This caused a difference in the privileges that were being allocated on the master and the slave. The fix ensures that creater of the stored routine is added to the binary log, and the slave now checks first if the user exists before granting privileges. To maintain compatibility with previous versions, the DEFINER is used when the INVOKER is not available. As part of this fix, anonymous users can be used to replicate from master to slave. (Bug #20049894)

Replication: If the I/O thread on a replication slave failed while it was in the initialization phase, it was not providing this information in the Last_IO_Error field after issuing SHOW SLAVE STATUS. The fix ensures that such errors are reported correctly in the Last_IO_Error field. (Bug #18909984)

Replication: After issuing RESET SLAVE, the RECEIVED_TRANSACTION_SET field in the performance_schema.replication_connection_status table showed incorrect values. This could cause an incorrect string value error. (Bug #18751585, Bug #19840342)

Replication: When gtid_mode=ON and slave_net_timeout was set to a low value, the slave I/O thread could appear to hang. This was due to the slave heartbeat not being sent regularly enough when the dump thread found many events that could be skipped. The fix ensures that the heartbeat is sent correctly in such a situation. (Bug #74607, Bug #19975697)

CMake failed to detect the OpenSSL version properly for recent versions of OpenSSL (the format of the version string changed). (Bug #20756770)

GRANT and ALTER USER could clear the password-expiration flag for operations not related to resetting the password. (Bug #20634154)

For upgrades from MySQL 5.6 to 5.7 that involve moving mysql.user table passwords from the Password column to the authentication_string column, mysql_upgrade neglected to handle rows with an empty plugin value and a pre-4.1 password hash. (Bug #20614545)

For table-modifying statements, the parser could dereference the parse tree without checking for out-of-memory conditions or null pointers. (Bug #20607407)

mysql_stmt_prepare() could leak memory allocated to metadata. (Bug #20598261)

Debian packages were missing some dependencies. (Bug #20561621)

A server exit could be caused by a query that contained a HAVING clause, which itself contained an IN() subquery predicate, where the subquery referenced a column of the query. (Bug #20558891)

The server could exit if a client using the cleartext authentication plugin attempted to connect with an empty password. (Bug #20537246)

A query cache invalidation function used a too-small buffer for holding encoded database names, which could result in a server exit. (Bug #20528928)

Valgrind warnings were silenced for display of GTID-related debug information. (Bug #20506672)

Some queries that had a derived table (subquery) in the FROM clause could raise an assertion. (Bug #20487336)

A table-modifying statement that followed a failed table-modifying could result in a server exit. (Bug #20460208)

Union queries over views containing ENUM or SET values were not handled properly. (Bug #20456178)

A natural left join between a derived table and a regular table, joined with another natural left join to another regular table could cause a server exit. (Bug #20455184)

The optimizer could try to create an index of the wrong data type on internal temporary tables. (Bug #20454833)

A multiple-table UPDATE statement where one of the specified tables was a derived table could cause a server exit. (Bug #20454533)

Mishandling of SRID values within ST_GeomFromGeoJSON() could cause an assertion to be raised. (Bug #20416705)

Under certain conditions, LCASE(), DECODE(), and ENCODE() could have source and destination overlap in memory-copying operations. (Bug #20315088, Bug #75931, Bug #20554017)

ST_Distance() could return incorrect results on 32-bit platforms. (Bug #20259578)

If a view was processed using the MERGE algorithm and had an ORDER BY clause, an error occurred if the view was queried using GROUP BY with the ONLY_FULL_GROUP_BY SQL mode enabled, unless the query selected all view columns. (Bug #20210742)

For debug builds, the optimizer could reject use of LooseScan for eq_ref access joins and raise an assertion. The optimizer now permits this combination for query execution. (Bug #20119743)

An out-of-range error in a subquery could raise an assertion. (Bug #20035071)

Renaming the mysql.procs_priv table and executing SHOW GRANTS resulted in a server exit. (Bug #20006361)

Ordering by a GROUP_CONCAT() result could cause a server exit. (Bug #19880368, Bug #20730220)

The server could exit due to an inappropriate full-text lookup using a full-text predicate within a subquery that contained an outer reference. (Bug #19828320)

For a prepared statement with an ORDER BY that refers by column number to a GROUP_CONCAT() expression that has an outer reference, repeated statement execution could cause a server exit. (Bug #19814337)

For a materialized internal temporary table used with semijoins, the optimizer could add an index to it but then use an inappropriate lookup strategy, causing a server exit. (Bug #19695490, Bug #21782943)

The optimizer could raise an assertion due to incorrectly associating an incorrect field with a temporary table. (Bug #19612819, Bug #20730129)

Specifying --general_log_file= (with an empty value) at server startup caused the server to fail and exit. (Bug #19392264)

Improper propagation of ORDER BY for a derived table or view used within a multiple-table UPDATE could raise an assertion. (Bug #18439019)

The thd_proc_info() function defined in plugin.h was not actually implemented. This has been changed to set_thd_proc_info(). (Bug #11844974)

For debug builds, an assertion was raised when calculating the symmetric difference between a MultiLineString and a MultiPoint. (Bug #77580, Bug #21355906)

The query rewrite framework introduced in MySQL 5.7.6 produced excesssive mutex acquisition that caused performance degradation under some conditions. (Bug #76509, Bug #20785598)

mysql_install_db started mysqld in bootstrap mode, but failed to wait for it to finish, causing premature shutdown and the need for crash recovery. (Bug #76344, Bug #20728488)

SHOW CREATE USER did not work for clients older than MySQL 5.7. (Bug #76093, Bug #20627890)

Inappropriate -Werror options could appear in mysql_config --cflags output. (Bug #76019, Bug #20590904)

With InnoDB as the default temporary table storage engine, InnoDB sometimes made incorrect assumptions about temporary table key part lengths. (Bug #76016, Bug #20590162)

Selecting from the global_variables Performance Schema table resulted in a spurious warning about the sql_log_bin system variable. (Bug #75980, Bug #20575529)

For the embedded server, proper deprecation warning were not produced for SHOW VARIABLES and SHOW STATUS statements that included a WHERE clause. (Bug #75951, Bug #20559828)

A subquery that contained a user-defined variable could cause an assertion to be raised. (Bug #75934, Bug #20554585)

AddressSanitizer compilation errors were silenced. (Bug #75739, Bug #20459338, Bug #75740, Bug #20459363)

Corrections were made for a number of code issues that resulted in compiler warnings about array bounds, possibly uninitialized variables, and variables being set but not used. (Bug #75735, Bug #20458574)

The mysql client could exit prematurely when invoked with the --quick option. (Bug #74182, Bug #19723750)

CHECK TABLE ... FOR UPGRADE did not report temporal columns that use the old datetime format (from before MySQL 5.6.4). Consequently, mysql_upgrade did not know to issue REPAIR TABLE statements to rebuild tables that contain such columns, and subsequent ALTER TABLE statements were unable to perform fast alterations to the extent possible had the tables been repaired. Now, if the avoid_temporal_upgrade system variable is disabled, CHECK TABLE reports old temporal columns and REPAIR TABLE upgrades tables from old temporal format to the new format. (Bug #73008, Bug #18985579)

With the offline_mode system variable enabled, the server sometimes failed to accept connection from a user with the SUPER privilege due to a race condition. (Bug #72760, Bug #18842228)

Information written to the slow query log for HANDLER ... READ statements always had rows_sent and rows_examined values of 0. (Bug #71892, Bug #18335504)

mysql_real_connect() could close a file descriptor twice if the server was not running. (Bug #69423, Bug #19226740)

Some key descriptors used by the optimizer were uninitialized. Thanks to Sergei Glushchenko for the patch. (Bug #68713, Bug #16512701)

EXPLAIN could show incorrect filtered values for queries that included a LIMIT clause. (Bug #34124, Bug #11747810)