Indexing ACF Fields

[ypantig]

Hi there,

I added custom field data into the record to be indexed and now the record size is bigger than the quota that was set. Is there anyway to go about this? I've reached out to Algolia and they suggested using distinct but since it doesn't look like there's a hook to do it (and it works via JS rather than PHP), not sure how I can go about doing so?

The record sizes I have vary, there are some that go below 10KB, but most of them go over. I have removed some data from the index, but it would be great of all the fields can be indexed (we're using a lot of flexible fields throughout the site and each page has a lot of layouts set).

Algolia couldn't increase the size and suggested reaching out to you guys.

Thanks.

[ypantig]

I was that I can adjust the indexing settings and distinct is set to true by default. Here's a screenshot of the error that I'm getting.

Any ideas on how to optimize the data? I'm adding the custom fields data in a separate attribute through algolia_searchable_post_shared_attributes

Here's the function that I have:

<?php
public function algolia__addACFFields( $attributes, $post ) {

    $postID = $post->ID;
    $template = get_page_template_slug( $postID );
    $template = str_replace( 'templates/', '', $template );
    $template = str_replace( '.php', '', $template );

    $content = '';

    switch ( $template )
    {
      case 'take-action':

        $fields = array( 'feature_action', 'act_online', 'act_locally', 'clean_living_tips', 'volunteer' );

        foreach ( $fields as $key => $value )
        {
          $group = get_field( $value, $postID );
          if ( isset( $group[ 'headline' ] ) && !empty( $group[ 'headline' ] ) )
          {
            $content .= $group[ 'headline' ] . ' ';
          }

          if ( isset( $group[ 'title' ] ) && !empty( $group[ 'title' ] ) )
          {
            $content .= $group[ 'title' ] . ' ';
          }

          $content .= strip_tags( $group[ 'content' ] ) . ' ';
        }

        break;

      case 'experts':

        $experts = get_field( 'experts_listing', $postID );

        if ( empty( $experts ) )
        {
          continue;
        }

        $experts = $experts[0][ 'experts' ];

        foreach ( $experts as $key => $expert )
        {
          $args[ 'term_taxonomy_id' ] = $expert;
          $term = get_term( $expert, 'expert' );

          $content .= $term->name . ' ' . $term->description . ' ';
        }

        break;

      default:
        // code...
        break;
    }

    if ( !empty( $content ) )
    {
      $attributes[ 'extra_content' ] = $content;
    }

    if ( get_post_type( $postID ) != 'page' )
    {
      return $attributes;
    }

    $blocks = get_field( 'blocks', $postID );

    if ( !empty( $blocks ) )
    {
      $blockContent = '';

      foreach ( $blocks as $key => $block )
      {
        $layout = $block[ 'acf_fc_layout' ];

         if ( isset( $block[ 'title' ] ) && !empty( $block[ 'title' ] ) )
         {
           $blockContent .= $block[ 'title' ] . ' ';
         }

         if ( isset( $block[ 'content' ] ) && !empty( $block[ 'content' ] ) )
         {
           $blockContent .= strip_tags( $block[ 'content' ] ) . ' ';
         }

         if ( isset( $block[ 'posts' ] ) && !empty( $block[ 'posts' ] ) )
         {
           foreach ( $block[ 'posts' ] as $key => $value )
           {
             $blockContent .= get_the_title( $value->ID ) . ' ';
             $blockContent .= get_the_excerpt( $value->ID );
           }
        }

        switch ( $layout ) {
          case 'card_columns':

            if ( isset( $block[ 'cards' ] ) && !empty( $block[ 'cards' ] ) )
            {
              foreach ( $block[ 'cards' ] as $key => $value )
              {
                $blockContent .= $value[ 'title' ];
                $blockContent .= strip_tags( $value[ 'content' ] );
              }
            }

            break;

          case 'by_the_numbers':
          case 'content_image':

            if ( isset( $block[ 'numbers' ] ) && !empty( $block[ 'numbers' ] ) )
            {
              foreach ( $block[ 'numbers' ] as $key => $value )
              {
                $blockContent .= $value[ 'title' ];
                $blockContent .= strip_tags( $value[ 'content' ] );
              }
            }

            break;

          case 'accordion':

            if ( isset( $block[ 'accordions' ] ) && !empty( $block[ 'accordions' ] ) )
            {
              foreach ( $block[ 'accordions' ] as $key => $value )
              {
                $blockContent .= $value[ 'title' ];
                $blockContent .= strip_tags( $value[ 'content' ] );
              }
            }

            break;

          default:
            // default
            break;

        }
      }

      if ( !empty( $blockContent ) )
      {
        $attributes[ 'extra_content' ] .= $blockContent;
      }
    }

    return $attributes;

  }

[rayrutjes]

Hi @ypantig ,

The plugin uses the distinct feature by default: https://community.algolia.com/wordpress/index-schema.html

However it uses distinct on the content field, which in general is the field causing issues.

In your case you might want to truncate the content of your attribute.

Can you tell me more about the attribute causing issues, what it contains?

[ypantig]

Hi @rayrutjes,

Thanks for getting back to me. I've added more info above. As well, I tried appending the custom field content to content and it's not getting appended for some reason.. Any ideas..? In the above function, I changed extra_content to say content and made sure that it said $attributes[ 'content' ] .= $content; on both instances

[ypantig]

Even if I try to do $attributes[ 'content' ] = $content;, the content is still not getting appended. I know there's content for a specific page the I'm testing on since the extra_content attribute is not empty before I did the switch to content.

[rayrutjes]

What is the hook you are using?
.
I think the reason why your content is not being append is because it has already been splitted.

Maybe you could try to append the given content to the native wp_content hook we are using here: https://github.com/algolia/algoliasearch-wordpress/blob/master/includes/indices/class-algolia-searchable-posts-index.php#L82

Right after that line, content gets exploded accross multiple records so if you want to change the content, it needs to happen before.
We might want to introduce a hook to make this simpler though.

Feel free to submit a PR here.

[ypantig]

I may have found the issue. I think I'm missing some fields in the function, which for some reason I wasn't when I was using the extra_content attribute that I added in. I'll keep you posted.

[ypantig]

Quick question, so I'm using the algolia_searchable_post_shared_attributes filter. I should be able to check what the page is using as a template correct..? That's what I'm having an issue on now (please let me know if you'd like me to create a new issue for this one).

So before, with the extra_content attribute, it's going through the switch statement I had up there. But for some reason now when adding to content, it's not appending it..

[ypantig]

I did some error logging and it looks like the content is being added to the attributes for the templates, but not getting to Algolia.. Here's a screenshot:

Any thoughts? It looks like this is just for fields from custom templates where I'm doing a check for the template. For a more general custom field, the content gets appended.

Here's a sample screenshot of a page entry in Algolia with content coming from that general custom field I was talking about earlier.

[ypantig]

Hi @rayrutjes any thoughts on the above?

[ypantig]

Hi @rayrutjes,

I was able to fix it by adding 3 lines of code in the plugin. I'll submit a PR as you suggested.

Here's the lines of code that I added in:

<?php 
if ( !empty( $shared_attributes[ 'content' ] ) ) {
    $post_content .= apply_filters( 'the_content', $shared_attributes[ 'content' ] );
}
?>

[rayrutjes]

Thanks @ypantig ! I made a comment on the PR.
Really appreciate you tackled this!