Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

semantic_text field mapper and inference - Partial PR #12

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
9332ef9
Merge branch 'main' into feature/semantic-text
Mikep86 Jan 12, 2024
9311f50
Merge branch 'main' into feature/semantic-text
Mikep86 Jan 17, 2024
f86ae02
Merge branch 'main' into feature/semantic-text
Mikep86 Jan 17, 2024
d06038c
Merge branch 'main' into feature/semantic-text
Mikep86 Jan 17, 2024
833469c
Store semantic_text model info in mappings (#103319)
Mikep86 Jan 17, 2024
64b4799
semantic_text inference results indexing (#103978)
Mikep86 Jan 18, 2024
eda88d0
Merge branch 'main' into feature/semantic-text
carlosdelest Jan 18, 2024
94805a6
Merge remote-tracking branch 'origin/main' into feature/semantic-text
carlosdelest Feb 1, 2024
551fe80
Merge remote-tracking branch 'origin/main' into feature/semantic-text
carlosdelest Feb 1, 2024
7e2610b
Merge remote-tracking branch 'origin/main' into feature/semantic-text
carlosdelest Feb 2, 2024
e3b6a65
Move semantic_text field mappers to inference plugin (#105187)
carlosdelest Feb 6, 2024
553484c
Merge remote-tracking branch 'origin/main' into feature/semantic-text
carlosdelest Feb 7, 2024
ca65a70
semantic_text - Field inference (#103697)
carlosdelest Feb 9, 2024
16762be
Merge remote-tracking branch 'origin/main' into feature/semantic-text
carlosdelest Feb 14, 2024
f3d5a78
Merge remote-tracking branch 'origin/main' into feature/semantic-text
carlosdelest Feb 20, 2024
ffa4d40
Merge remote-tracking branch 'origin/main' into feature/semantic-text
carlosdelest Feb 21, 2024
3f7ccde
Merge remote-tracking branch 'origin/main' into feature/semantic-text
carlosdelest Mar 5, 2024
881c394
Merge remote-tracking branch 'origin/main' into feature/semantic-text
carlosdelest Mar 5, 2024
b1a3ee8
Semantic text dense vector support (#105515)
carlosdelest Mar 6, 2024
2039fb3
This was supposed to be merged into #105515 but didn't make it
carlosdelest Mar 6, 2024
db67976
Merge branch 'main' into feature/semantic-text
Mikep86 Mar 18, 2024
3ca808b
semantic_text - extract Index Metadata inference information to separ…
carlosdelest Mar 19, 2024
823fb58
[feature/semantic_text] Refactor inference to run as an action filter…
jimczi Mar 20, 2024
d4e283d
[feature/semantic_text] Register semantic text sub fields in the mapp…
jimczi Mar 22, 2024
9531948
Merge remote-tracking branch 'origin/main' into feature/semantic-text
carlosdelest Mar 27, 2024
122e439
Fix build error
Mikep86 Mar 27, 2024
ef3abd9
[feature/semantic-text] Simplify the integration of the field inferen…
jimczi Mar 28, 2024
2e89d99
Merge branch 'main' into feature/semantic-text
Mikep86 Mar 28, 2024
b6ca8d2
[feature/semantic-text] semantic text copy to support (#106689)
carlosdelest Apr 2, 2024
2c11a3f
Merge remote-tracking branch 'upstream/main' into feature/semantic-text
jimczi Apr 4, 2024
5556763
[feature/semantic-text] Move the inference results back to the origin…
jimczi Apr 5, 2024
6b83424
Merge branch 'refs/heads/main' into feature/semantic-text
carlosdelest Apr 5, 2024
f565596
[feature/semantic-text] Handle chunked error (#107192)
jimczi Apr 8, 2024
937572d
Add first query tests
carlosdelest Apr 9, 2024
bef2214
Add inner_hits tests
carlosdelest Apr 9, 2024
84a2735
Add mapping incompatibility tests
carlosdelest Apr 9, 2024
81c864c
Merge remote-tracking branch 'origin/main' into feature/semantic-text
carlosdelest Apr 9, 2024
750f895
Merge branch 'refs/heads/feature/semantic-text' into carlosdelest/sem…
carlosdelest Apr 9, 2024
1f78a4a
Merge branch 'refs/heads/main' into carlosdelest/semantic-text-field-…
carlosdelest Apr 9, 2024
b0e6d43
Add semantic_text field mapper and inference generation
carlosdelest Apr 9, 2024
537f610
Merge branch 'refs/heads/carlosdelest/semantic-text-add-query-tests' …
carlosdelest Apr 9, 2024
c1ef120
[Docs][ESQL] Make functions reference more digestible (#107258)
leemthompo Apr 9, 2024
3dccc66
[DOCS][ESQL] Rename _Learning ESQL_ to _ESQL reference_ (#107259)
leemthompo Apr 9, 2024
73e8984
Add documentation to thread pool and executor code (#106208)
DiannaHohensee Apr 9, 2024
3bce501
Add tests pending from #107256
carlosdelest Apr 9, 2024
9ae414e
ESQL: Merge randomize a test attribute (#107239)
nik9000 Apr 9, 2024
c50fcb9
[DOCS] Expands the list of possible values of the result parameter of…
szabosteve Apr 9, 2024
62f19e3
Disable dynamic pruning on unindexed fields. (#107194)
jpountz Apr 9, 2024
3674a6f
ESQL: Commit LocateNoStartEvaluator (#107270)
alex-spies Apr 9, 2024
4a5329d
typo: "not found" -> "not_found" (#107276)
seanstory Apr 9, 2024
3c29dcb
Merge branch 'refs/heads/carlosdelest/semantic-text-index-metadata-ch…
carlosdelest Apr 9, 2024
0f57a5b
Fix merge
carlosdelest Apr 9, 2024
3e847f4
Fix merge
carlosdelest Apr 9, 2024
c0960fa
Fix merge
carlosdelest Apr 9, 2024
31c05e9
ESQL: allow sorting by expressions and not only regular fields (#107158)
astefan Apr 9, 2024
aba7566
ESQL: Better tests to AUTO_BUCKET (#107228)
nik9000 Apr 9, 2024
24aed5c
Simplify merging enrich output (#107018)
dnhatn Apr 9, 2024
75228df
Add granular error list to alias action response (#106514)
parkertimmins Apr 9, 2024
8852566
Move ESQL's LOCATE test cases to cases (#107271)
nik9000 Apr 9, 2024
96227a1
ESQL: Generate kibana inline docs (#106782)
nik9000 Apr 9, 2024
12398ee
Don't overwrite `DataStream.rolloverOnWrite` flag on failure store ro…
nielsbauman Apr 9, 2024
6ff6dc1
[Transform] Only trigger action once per thread (#107232)
prwhelan Apr 9, 2024
13f95fd
[Profiling] Divide into more packages (#107201)
danielmitterdorfer Apr 10, 2024
ec2a4ca
Expand release note for #105044 (#107257)
DaveCTurner Apr 10, 2024
de171b8
Use merge sort instead of hashing to avoid performance issues with ma…
iverase Apr 10, 2024
6507ba5
[Profiling] Limit TopN functions to available data (#107296)
danielmitterdorfer Apr 10, 2024
e21f2e3
[Transform] Make force-stopping the transform always remove persisten…
przemekwitek Apr 10, 2024
c4a11de
Make API key actions local-only (#107148)
n1v0lg Apr 10, 2024
62729c9
Port DocsTest gradle plugin to java (#107124)
breskeby Apr 10, 2024
d6f9d1e
ESQL: Rename AUTO_BUCKET to just BUCKET (#107197)
bpintea Apr 10, 2024
ceeee1b
Recommend NFS over S3-like repositories (#107297)
DaveCTurner Apr 10, 2024
8bcbc97
Rename generated docs for (renamed) BUCKET func (#107299)
bpintea Apr 10, 2024
84d6157
Do not report document metering on system indices (#107041)
pgomulka Apr 10, 2024
b3bcc81
[Transform] Release test resources (#107057)
prwhelan Apr 10, 2024
8638dee
Openai model_id is required (#107286)
jonathan-buttner Apr 10, 2024
943885d
[DOCS][ESQL] Render locate function docs (#107305)
leemthompo Apr 10, 2024
19e9fc3
ES|QL: regex warnings in csv-spec tests (#107273)
luigidellaquila Apr 10, 2024
48a88c5
Renaming GeoIpDownloaderStatsAction (#107290)
masseyke Apr 10, 2024
a9cab35
Log skipped elections due to shutdown marker (#106701)
DiannaHohensee Apr 10, 2024
c57dd98
semantic_text: Add index metadata information for inference field map…
carlosdelest Apr 10, 2024
9e502aa
Expanding and refactoring the vector rolling upgrade tests (#107020)
benwtrent Apr 10, 2024
ff8365a
Update docs/changelog/107262.yaml
carlosdelest Apr 10, 2024
7d8fe11
Merge branch 'refs/heads/main' into carlosdelest/semantic-text-field-…
carlosdelest Apr 10, 2024
6805726
Merge remote-tracking branch 'carlosdelest/carlosdelest/semantic-text…
carlosdelest Apr 10, 2024
7dbb53b
Update changelog
carlosdelest Apr 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/

package org.elasticsearch.gradle.internal.doc

import org.elasticsearch.gradle.fixtures.AbstractGradleInternalPluginFuncTest
import org.elasticsearch.gradle.internal.conventions.precommit.PrecommitPlugin
import org.gradle.testkit.runner.TaskOutcome

class DocsTestPluginFuncTest extends AbstractGradleInternalPluginFuncTest {
Class<? extends PrecommitPlugin> pluginClassUnderTest = DocsTestPlugin.class

def setup() {
File docDir = new File(projectDir, 'doc');
docDir.mkdirs()
addSampleDoc(docDir)
buildApiRestrictionsDisabled = true
configurationCacheCompatible = false;
buildFile << """
tasks.named('listSnippets') {
docs = fileTree('doc')
}
tasks.named('listConsoleCandidates') {
docs = fileTree('doc')
}
"""
}

def "can list snippets"() {
when:
def result = gradleRunner("listSnippets").build()
then:
result.task(":listSnippets").outcome == TaskOutcome.SUCCESS
assertOutputContains(result.output, """
> Task :listSnippets
mapper-annotated-text.asciidoc[37:39](Painless)
mapper-annotated-text.asciidoc[42:44](js)
mapper-annotated-text.asciidoc[51:69](console)// TEST[setup:seats]
""")
}

def "can console candidates"() {
when:
def result = gradleRunner("listConsoleCandidates").build()
then:
result.task(":listConsoleCandidates").outcome == TaskOutcome.SUCCESS
assertOutputContains(result.output, """
> Task :listConsoleCandidates
mapper-annotated-text.asciidoc[42:44](js)
""")
}

void addSampleDoc(File docFolder) {
new File(docFolder, "mapper-annotated-text.asciidoc").text = """
[[painless-filter-context]]
=== Filter context
Use a Painless script as a {ref}/query-dsl-script-query.html[filter] in a
query to include and exclude documents.
*Variables*
`params` (`Map`, read-only)::
User-defined parameters passed in as part of the query.
`doc` (`Map`, read-only)::
Contains the fields of the current document where each field is a
`List` of values.
*Return*
`boolean`::
Return `true` if the current document should be returned as a result of
the query, and `false` otherwise.
*API*
The standard <<painless-api-reference-shared, Painless API>> is available.
*Example*
To run this example, first follow the steps in
<<painless-context-examples, context examples>>.
This script finds all unsold documents that cost less than \$25.
[source,Painless]
----
doc['sold'].value == false && doc['cost'].value < 25
----
[source,js]
----
curl 'hello world'
----
Defining `cost` as a script parameter enables the cost to be configured
in the script query request. For example, the following request finds
all available theatre seats for evening performances that are under \$25.
[source,console]
----
GET seats/_search
{
"query": {
"bool": {
"filter": {
"script": {
"script": {
"source": "doc['sold'].value == false && doc['cost'].value < params.cost",
"params": {
"cost": 25
}
}
}
}
}
}
}
----
// TEST[setup:seats]
"""
}
}

This file was deleted.

Loading
Loading