BUG: Not able to search meta fields on multisite setup

[moritzlang]

Describe the bug
When using Elasticpress in a WP multisite setup meta fields are not included in the search fields.
Searching for the post title does work though.

Steps to Reproduce

1. Setup ep_prepare_meta_data filter and sync the content.

add_filter("ep_prepare_meta_data", function ($meta, $post) {
    $meta = [];
    $meta["headlines"] = "Foo";
    return $meta;
}, 10, 2);

2. Do a search using WP_Query.

  $posts = get_posts(array(
    "ep_integrate" => true,
    "post_type" => array("page", "news", "downloads"),
    "s" => "Foo",
    "posts_per_page" => -1,
    "search_fields" => array(
      "post_title",
      "post_content",
      "post_excerpt",
      "text",
      "meta" => array("headlines"),
    ),
  ));

3. No results are returned, although the search query string is present in the meta "headlines" field.

I realized that the elasticpress search does not use the meta fields e.g. meta.headlines.value. See this logs:

{
   "type":"index_search_slowlog",
   "timestamp":"2021-12-02T15:40:34,306Z",
   "level":"WARN",
   "component":"i.s.s.fetch",
   "cluster.name":"docker-cluster",
   "node.name":"748b7c953fef",
   "message":"[REDACTED][4]",
   "took":"148.4micros",
   "took_millis":"0",
   "total_hits":"0 hits",
   "stats":"[]",
   "search_type":"QUERY_THEN_FETCH",
   "total_shards":"5",
   "source":"{\"from\":0,\"size\":10000,\"query\":{\"function_score\":{\"query\":{\"bool\":{\"should\":[{\"bool\":{\"must\":[{\"bool\":{\"should\":[{\"multi_match\":{\"query\":\"Foo\",\"fields\":[\"post_author.display_name^1.0\",\"post_content^1.0\",\"post_excerpt^1.0\",\"post_title^1.0\",\"terms.ep_custom_result.name^9999.0\"],\"type\":\"phrase\",\"operator\":\"OR\",\"slop\":0,\"prefix_length\":0,\"max_expansions\":50,\"zero_terms_query\":\"NONE\",\"auto_generate_synonyms_phrase_query\":true,\"fuzzy_transpositions\":true,\"boost\":4.0}},{\"multi_match\":{\"query\":\"Foo\",\"fields\":[\"post_author.display_name^1.0\",\"post_content^1.0\",\"post_excerpt^1.0\",\"post_title^1.0\",\"terms.ep_custom_result.name^9999.0\"],\"type\":\"best_fields\",\"operator\":\"AND\",\"slop\":0,\"fuzziness\":\"0\",\"prefix_length\":0,\"max_expansions\":50,\"zero_terms_query\":\"NONE\",\"auto_generate_synonyms_phrase_query\":true,\"fuzzy_transpositions\":true,\"boost\":2.0}},{\"multi_match\":{\"query\":\"Foo\",\"fields\":[\"post_author.display_name^1.0\",\"post_content^1.0\",\"post_excerpt^1.0\",\"post_title^1.0\"],\"type\":\"best_fields\",\"operator\":\"OR\",\"slop\":0,\"fuzziness\":\"1\",\"prefix_length\":0,\"max_expansions\":50,\"zero_terms_query\":\"NONE\",\"auto_generate_synonyms_phrase_query\":true,\"fuzzy_transpositions\":true,\"boost\":1.0}}],\"adjust_pure_negative\":true,\"boost\":1.0}}],\"filter\":[{\"match\":{\"post_type.raw\":{\"query\":\"page\",\"operator\":\"OR\",\"prefix_length\":0,\"max_expansions\":50,\"fuzzy_transpositions\":true,\"lenient\":false,\"zero_terms_query\":\"NONE\",\"auto_generate_synonyms_phrase_query\":true,\"boost\":1.0}}}],\"adjust_pure_negative\":true,\"boost\":1.0}},{\"bool\":{\"must\":[{\"bool\":{\"should\":[{\"multi_match\":{\"query\":\"Foo\",\"fields\":[\"post_author.display_name^1.0\",\"post_content^1.0\",\"post_excerpt^1.0\",\"post_title^1.0\",\"terms.category.name^1.0\",\"terms.ep_custom_result.name^9999.0\"],\"type\":\"phrase\",\"operator\":\"OR\",\"slop\":0,\"prefix_length\":0,\"max_expansions\":50,\"zero_terms_query\":\"NONE\",\"auto_generate_synonyms_phrase_query\":true,\"fuzzy_transpositions\":true,\"boost\":4.0}},{\"multi_match\":{\"query\":\"Foo\",\"fields\":[\"post_author.display_name^1.0\",\"post_content^1.0\",\"post_excerpt^1.0\",\"post_title^1.0\",\"terms.category.name^1.0\",\"terms.ep_custom_result.name^9999.0\"],\"type\":\"best_fields\",\"operator\":\"AND\",\"slop\":0,\"fuzziness\":\"0\",\"prefix_length\":0,\"max_expansions\":50,\"zero_terms_query\":\"NONE\",\"auto_generate_synonyms_phrase_query\":true,\"fuzzy_transpositions\":true,\"boost\":2.0}},{\"multi_match\":{\"query\":\"Foo\",\"fields\":[\"post_author.display_name^1.0\",\"post_content^1.0\",\"post_excerpt^1.0\",\"post_title^1.0\",\"terms.category.name^1.0\"],\"type\":\"best_fields\",\"operator\":\"OR\",\"slop\":0,\"fuzziness\":\"1\",\"prefix_length\":0,\"max_expansions\":50,\"zero_terms_query\":\"NONE\",\"auto_generate_synonyms_phrase_query\":true,\"fuzzy_transpositions\":true,\"boost\":1.0}}],\"adjust_pure_negative\":true,\"boost\":1.0}}],\"filter\":[{\"match\":{\"post_type.raw\":{\"query\":\"news\",\"operator\":\"OR\",\"prefix_length\":0,\"max_expansions\":50,\"fuzzy_transpositions\":true,\"lenient\":false,\"zero_terms_query\":\"NONE\",\"auto_generate_synonyms_phrase_query\":true,\"boost\":1.0}}}],\"adjust_pure_negative\":true,\"boost\":1.0}},{\"bool\":{\"must\":[{\"bool\":{\"should\":[{\"multi_match\":{\"query\":\"Foo\",\"fields\":[\"post_author.display_name^1.0\",\"post_content^1.0\",\"post_excerpt^1.0\",\"post_title^1.0\",\"terms.ep_custom_result.name^9999.0\"],\"type\":\"phrase\",\"operator\":\"OR\",\"slop\":0,\"prefix_length\":0,\"max_expansions\":50,\"zero_terms_query\":\"NONE\",\"auto_generate_synonyms_phrase_query\":true,\"fuzzy_transpositions\":true,\"boost\":4.0}},{\"multi_match\":{\"query\":\"Foo\",\"fields\":[\"post_author.display_name^1.0\",\"post_content^1.0\",\"post_excerpt^1.0\",\"post_title^1.0\",\"terms.ep_custom_result.name^9999.0\"],\"type\":\"best_fields\",\"operator\":\"AND\",\"slop\":0,\"fuzziness\":\"0\",\"prefix_length\":0,\"max_expansions\":50,\"zero_terms_query\":\"NONE\",\"auto_generate_synonyms_phrase_query\":true,\"fuzzy_transpositions\":true,\"boost\":2.0}},{\"multi_match\":{\"query\":\"Foo\",\"fields\":[\"post_author.display_name^1.0\",\"post_content^1.0\",\"post_excerpt^1.0\",\"post_title^1.0\"],\"type\":\"best_fields\",\"operator\":\"OR\",\"slop\":0,\"fuzziness\":\"1\",\"prefix_length\":0,\"max_expansions\":50,\"zero_terms_query\":\"NONE\",\"auto_generate_synonyms_phrase_query\":true,\"fuzzy_transpositions\":true,\"boost\":1.0}}],\"adjust_pure_negative\":true,\"boost\":1.0}}],\"filter\":[{\"match\":{\"post_type.raw\":{\"query\":\"downloads\",\"operator\":\"OR\",\"prefix_length\":0,\"max_expansions\":50,\"fuzzy_transpositions\":true,\"lenient\":false,\"zero_terms_query\":\"NONE\",\"auto_generate_synonyms_phrase_query\":true,\"boost\":1.0}}}],\"adjust_pure_negative\":true,\"boost\":1.0}}],\"adjust_pure_negative\":true,\"boost\":1.0}},\"functions\":[{\"filter\":{\"match_all\":{\"boost\":1.0}},\"exp\":{\"post_date_gmt\":{\"scale\":\"14d\",\"decay\":0.25,\"offset\":\"7d\"},\"multi_value_mode\":\"MIN\"}},{\"filter\":{\"match_all\":{\"boost\":1.0}},\"weight\":0.001}],\"score_mode\":\"sum\",\"boost_mode\":\"multiply\",\"max_boost\":3.4028235E38,\"boost\":1.0}},\"post_filter\":{\"bool\":{\"must\":[{\"terms\":{\"post_type.raw\":[\"page\",\"news\",\"downloads\"],\"boost\":1.0}},{\"term\":{\"post_status\":{\"value\":\"publish\",\"boost\":1.0}}}],\"adjust_pure_negative\":true,\"boost\":1.0}},\"sort\":[{\"post_date\":{\"order\":\"desc\"}}]}",
   "cluster.uuid":"AtaZXep6S2KvLh85ncTVEw",
   "node.id": "3aS0sOI4Q22rLceizEMPiw"
 }

I also tried setting the search fields using this ep_search_fields filter (without success):

/**
* Modify available search fields
*/
add_filter('ep_search_fields', function($search_fields, $args) {
   $meta_fields = ['headlines'];
   foreach($meta_fields as $key) {
       $fields[] = 'meta.'.$key.'.value';
   }
   return array_merge( $search_fields, $fields );
}, 10,2 );

Expected behavior
Posts which contain the search query string in the meta fields should be returned in the WP_Query results.

Is this a multisite issue maybe?
What can I do to include the meta fields in the search fields?

Environment information

• Device: MacBook
• OS: MacOS 11.3
• Browser and version: Chrome 96.0.4664.55
• WordPress version: 5.8.1

• ElasticPress version:
• Elasticsearch version: 7.9
• Where do you host your Elasticsearch server? Docker
• What ElasticPress features do you have currently active? Post Search, Facets

Site Health Info:

### wp-core ###

version: 5.8.1
site_language: de_AT
user_language: en_US
timezone: +00:00
permalink: /%postname%/
https_status: false
multisite: true
user_registration: false
blog_public: 1
default_comment_status: open
environment_type: production
user_count: 84
site_count: 17
network_count: 1
dotorg_communication: true

wp-active-theme

name: Theme (theme/resources)
version: 9.0.10
author: Roots
author_website: https://roots.io/
parent_theme: none
theme_features: core-block-patterns, widgets-block-editor, soil-clean-up, soil-jquery-cdn, soil-nav-walker, soil-nice-search, soil-relative-urls, title-tag, menus, post-thumbnails, html5, customize-selective-refresh-widgets, editor-style, widgets
theme_path: /Users/mla/Documents/projects/REDACTED/public/app/themes/theme/resources

wp-mu-plugins (1)

WP Migrate DB Pro Compatibility: version: 1.2, author: Delicious Brains

wp-plugins-active (16)

Advanced Custom Fields: Extended PRO: version: 0.8.8.3, author: ACF Extended (latest version: 0.8.8.6)
Advanced Custom Fields: Multisite Select: version: 1.0.0, author: James H-Hall
Advanced Custom Fields PRO: version: 5.9.6, author: Elliot Condon (latest version: 5.11.3)
Classic Editor: version: 1.6.2, author: WordPress Contributors
ElasticPress: version: 3.6.5, author: 10up
PublishPress Capabilities: version: 2.2, author: PublishPress (latest version: 2.3)
PublishPress Permissions: version: 3.5.7, author: PublishPress (latest version: 3.6.5)
Required taxonomies: version: 1.1.7, author: VegaCorp
Smash Balloon Instagram Feed: version: 2.9.3.1, author: Smash Balloon (latest version: 2.9.7)
WP Migrate DB Pro: version: 2.2.0, author: Delicious Brains (latest version: 2.2.1)
WP Migrate DB Pro CLI: version: 1.6.0, author: Delicious Brains
WP Migrate DB Pro Media Files: version: 2.1.0, author: Delicious Brains
WP Migrate DB Pro Multisite Tools: version: 1.4.0, author: Delicious Brains (latest version: 1.4.1)
WP Migrate DB Pro Theme & Plugin Files: version: 1.2.0, author: Delicious Brains
Yoast Duplicate Post: version: 4.1.2, author: Enrico Battocchi & Team Yoast (latest version: 4.2)
Yoast SEO: version: 17.1, author: Team Yoast (latest version: 17.7.1)

wp-plugins-inactive (10)

ACF: FocusPoint: version: 1.1.8, author: Oskari Oksanen
Add From Server: version: 3.4.5, author: Dion Hulse
Advanced Custom Fields: Icon Picker: version: 1.9.0, author: Houke de Kwant
Cache Enabler: version: 1.8.4, author: KeyCDN (latest version: 1.8.7)
CDN Enabler: version: 2.0.4, author: KeyCDN
Multisite Post Duplicator: version: 1.7.6, author: Mario Jaconelli
Password Policy Manager: version: 1.1.4, author: miniOrange
Sucuri Security - Auditing, Malware Scanner and Hardening: version: 1.8.28, author: Sucuri Inc. (latest version: 1.8.30)
User Switching: version: 1.5.7, author: John Blackbourn & contributors (latest version: 1.5.8)
WP 2FA - Two-factor authentication for WordPress: version: 1.7.1, author: WP White Security

wp-media

image_editor: WP_Image_Editor_GD
imagick_module_version: Not available
imagemagick_version: Not available
imagick_version: Not available
file_uploads: File uploads is turned off
post_max_size: 512M
upload_max_filesize: 512M
max_effective_size: 512 MB
max_file_uploads: 20
gd_version: 2.3.2
gd_formats: GIF, JPEG, PNG, WebP, BMP, XPM
ghostscript_version: not available

wp-server

server_architecture: Darwin 20.4.0 x86_64
httpd_software: nginx/1.19.6
php_version: 7.4.20 64bit
php_sapi: fpm-fcgi
max_input_variables: 1000
time_limit: 30
memory_limit: 512M
max_input_time: 60
upload_max_filesize: 512M
php_post_max_size: 512M
curl_version: 7.78.0 (SecureTransport) OpenSSL/1.1.1k
suhosin: false
imagick_availability: false
pretty_permalinks: true

wp-database

extension: mysqli
server_version: 5.7.34
client_version: mysqlnd 7.4.20

wp-constants

WP_HOME: http://REDACTED.test
WP_SITEURL: http://REDACTED.test
WP_CONTENT_DIR: /Users/mla/Documents/projects/REDACTED/public/app
WP_PLUGIN_DIR: /Users/mla/Documents/projects/REDACTED/public/app/plugins
WP_MEMORY_LIMIT: 64M
WP_MAX_MEMORY_LIMIT: 512M
WP_DEBUG: true
WP_DEBUG_DISPLAY: true
WP_DEBUG_LOG: true
SCRIPT_DEBUG: false
WP_CACHE: false
CONCATENATE_SCRIPTS: undefined
COMPRESS_SCRIPTS: undefined
COMPRESS_CSS: undefined
WP_LOCAL_DEV: undefined
DB_CHARSET: utf8
DB_COLLATE: undefined

wp-filesystem

wordpress: writable
wp-content: writable
uploads: writable
plugins: writable
themes: writable
mu-plugins: writable

[felipeelia]

Hi @moritzlang, it seems you have some custom code to handle weighting, is that right? I don't think the plugin would add the weight like post_author.display_name^1.0. If that is the case you won't have it working with search_fields. As stated in that filter docs you'll have to use ep_weighting_configuration_for_search.

Something like this should do it:

function ep_2482_weighting_configuration_for_search( $weight_config, $args ) {
	$post_type = (array) $args['post_type'];
	if ( in_array( 'post', $post_type, true ) ) {
		$weight_config['post']['meta.headlines.value'] = array(
			'weight'  => 20,
			'enabled' => true, // Updated. It wrongly was '1' before.
		);
	}
	return $weight_config;
}
add_filter( 'ep_weighting_configuration_for_search', 'ep_2482_weighting_configuration_for_search', 10, 2 );

I'm closing the issue now but feel free to reopen if needed. Thanks!

[moritzlang]

Hey @felipeelia, thanks for the help!

I had time to get back to this problem, so here is my update:

1. There is no custom code for handling weighting in my codebase. But your suggested approach using ep_weighting_configuration_for_search did work! It took me some time to find out that "enabled" => 1 does not work, the value has to be set to true in order to update the $weight_config with the fields (unfortunately every documentation available shows an example using the wrong "enabled" => 1).

2. One problem is left: I want to search through all the sites of my multisite setup, but when setting sites => "all" no results are returned. Any way to debug/fix this?

[felipeelia]

Good catch on the 1 vs true, @moritzlang. It seems to come from this line: https://github.com/10up/ElasticPress/blob/develop/includes/classes/Feature/Search/Weighting.php#L538 It would be great if you could send us a PR making that code to accept both values or point us to the docs containing the wrong code (I've updated my comment now.)

For issue #2, Debug Bar and Debug Bar ElasticPress would be the way to go (more here). You may need to recreate the network alias using wp elasticpress recreate-network-alias. Give that a try and if it does not work I suggest you create a new issue sharing the ES query being sent and the results being returned.

Thanks!

[moritzlang]

@felipeelia Added PR #2597