Improve search process

- Add howto sources
- Don't show duplicate URL sources
- Also search based on user question

heymans/documentation.py

@@ -111,7 +111,7 @@ def search(self, queries):
             return []
         docs = []
         for query in queries:
-            logger.info(f'searching FAISS: {query}')
+            logger.info(f'searching FAISS')
             for doc in self._retriever.invoke(query):
                 doc_desc = f'{doc.metadata["url"]} ({doc.metadata["seq_num"]})'
                 if any(doc.page_content == ref.page_content for ref in docs):

heymans/heymans.py

@@ -100,6 +100,11 @@ def _search(self, message: str) -> GeneratorType:
                'message': f'{config.ai_name} is searching '}, {}
         logger.info('[search state] entering')
         self.documentation.clear()
+        # First seach based on the user question
+        logger.info('[search state] searching based on user message')
+        self.documentation.search([message])
+        # Then search based on the search-model queries derived from the user
+        # question
         self.tools = self.search_tools
         reply = self.search_model.predict(self.messages.prompt(
             system_prompt=prompt.SYSTEM_PROMPT_SEARCH))

heymans/routes/app.py

@@ -50,9 +50,9 @@ def chat_page():
         if 'sources' in metadata:
             sources_div = '<div class="message-sources">'
             sources = json.loads(metadata['sources'])
-            for source in sources:
-                if source['url']:
-                    sources_div += f'<a href="{source["url"]}">{source["url"]}</a><br />'
+            urls = {source['url'] for source in sources if source['url']}
+            for url in urls:
+                sources_div += f'<a href="{url}">{url}</a><br />'
             sources_div += '</div>'
         else:
             sources_div = ''

heymans/templates/main.js

@@ -178,9 +178,13 @@ async function sendMessage(message) {
         // Flag to track if there are any valid sources
         // console.log(metadata['sources']);
         let hasValidSources = false;
+        let uniqueURLs = new Set();
+
         sources.forEach(sourceObj => {
-            if (sourceObj.url) {
+            if (sourceObj.url && !uniqueURLs.has(sourceObj.url)) {
                 hasValidSources = true;
+                uniqueURLs.add(sourceObj.url);
+
                 const sourceLink = document.createElement('a');
                 sourceLink.href = sourceObj.url;
                 sourceLink.textContent = sourceObj.url;

sources/jsonl/opensesame-howtos.jsonl

@@ -0,0 +1,28 @@
+{"content": "To randomly select an image from a predefined list in an inline_script:\n\n1. In an INLINE_SCRIPT item, create a list of image file names, e.g., `image_list = [\"image1.png\", \"image2.png\", \"image3.png\"]`\n2. Use `random.choice(image_list)` to randomly select an image file name from the list\n3. Store the randomly selected image file name in an experimental variable, e.g., `chosen_image = random.choice(image_list)`\n4. In a SKETCHPAD item, draw a dummy image (which can be anything)\n4. Edit the script of the SKETCHPAD and use the `chosen_image` variable for the `file` keyword:\n    ```\n    draw image center=1 file=\"{chosen_image}\" scale=1 show_if=True x=0 y=0 z_index=0\n    ```", "url": "https://forum.cogsci.nl", "title": "How to randomly select an image from a list in an inline_script an show it in a SKETCHPAD"}
+{"content": "To measure response times for each option in a custom FORM_BASE item:\n\n1. FORM_BASE items don't automatically record a response time, so you need to manually track it.\n2. Insert a FORM_BASE item into your experiment. Let's assume it's called `my_form`.\n3. Insert an INLINE_SCRIPT item after the my_form.\n4. In the run phase of the INLINE_SCRIPT, calculate the response time by subtracting the onset time of `my_form` from the current time, like so:\n    ```python\n    response_time = clock.time() - time_my_form\n    ```", "url": "https://forum.cogsci.nl", "title": "How to measure response times in a custom FORM_BASE item"}
+{"content": "When an experiment fails to open correctly in OpenSesame, you can take the following steps to troubleshoot the issue:\n\n1. Check that you are using the most recent version of OpenSesame. If you are using an outdated version, the experiment may not be compatible.\n2. If you are using a recent version of OpenSesame, close OpenSesame completely and then re-open the experiment.\n3. Check the debug window (View \u2192 Show console) for error messages that may indicate why the experiment is failing to open.\n5. If you still cannot get the experiment to open correctly, try downloading the experiment again. The downloaded file may have been corrupted.\n6. If problems persist, contact the original author of the experiment for further assistance.", "url": "https://forum.cogsci.nl", "title": "How to troubleshoot an experiment that fails to open correctly"}
+{"content": "If you need help with OpenSesame, there are several places you can look:\n\n1. Consult the built-in help documentation by clicking the help buttons that appear throughout the interface, or by clicking Help in the main menu. This will open the documentation in your web browser.\n2. Visit the documentation website at [osdoc.cogsci.nl](https://osdoc.cogsci.nl). This site contains an extensive manual as well as several step-by-step tutorials.\n3. Watch video tutorials on the [YouTube channel](https://www.youtube.com/sebastiaanmathot). These videos cover many basic topics to help you get started with OpenSesame.\n4. Ask a question on the [OpenSesame forum](https://forum.cogsci.nl). This forum is the main place for getting personalized help with OpenSesame. Be sure to include enough details, such as a description of what you are trying to achieve, and screenshots of things that you need help with.\n5. Ask me! As an AI assistant with access to the OpenSesame documentation, I will be able to help you with many things.", "url": "https://forum.cogsci.nl", "title": "How to find help when working with OpenSesame"}
+{"content": "To share an experiment with others, so that they can run and/ or inspect the experiment themselves, you can take the following steps:\n\n1. In OpenSesame, open the experiment that you would like to share.\n2. Make sure that the experiment contains no sensitive information, such as participant data.\n3. Save the experiment and close it.\n4. Locate the experiment file (`.osexp`) on your computer.\n5. Upload this file to a file sharing service, such as Google Drive, Dropbox, or the Open Science Framework (OSF).\n6. Once the upload is complete, make sure that the file is shared with the intended recipients.\n7. Send the download link to the people with whom you want to share the experiment.\n8. The recipients can then download the experiment and open it in their own copy of OpenSesame.", "url": "https://forum.cogsci.nl", "title": "How to share an experiment with others"}
+{"content": "To import (open) an experiment into OpenSesame, for example an experiment that someone else has shared with you, take the following steps:\n\n1. Download the experiment file, which should have the `.osexp` extension, to your computer.\n2. Launch OpenSesame.\n3. In the menu, click on File \u2192 Open and browse to the location where you have downloaded the `.osexp` file.\n4. Select the `.osexp` file and click on 'Open'.\n5. The downloaded experiment will now open in OpenSesame.", "url": "https://forum.cogsci.nl", "title": "How to import (open) an experiment into OpenSesame"}
+{"content": "To get the coordinates of the last character in a Text element:\n\n1. Create a Text element on your canvas with the desired sentence except for the last character. So if you want the coordinates of the 'i' in 'sigmundai', you would use the string 'sigmunda'.\n2. Use the `boundingbox` property of the Text element to get the bounding box of the entire text string. The bounding box is a tuple of four values: (left, top, right, bottom).\n3. The x-coordinate of the last character (i.e. the 'i' in this example) will be approximately equal to the `right` value of the bounding box.\n4. To get the y-coordinate of the last character, you can use the `top` or `bottom` value of the bounding box, depending on whether you want the coordinate of the top or bottom of the character.\n5. If needed, you can adjust the x-coordinate by subtracting a small value to account for any padding on the right side of the text.\n\n```python\nmy_text = 'sigmundai'\nmy_canvas = Canvas()\nmy_canvas['sentence'] = Text(my_text[:-1])\nbbox = my_canvas['sentence'].rect", "url": "https://forum.cogsci.nl", "title": "How to get the coordinates of the last character in a Text element"}
+{"content": "last_char_x = bbox[2]", "url": "https://forum.cogsci.nl", "title": "important: not bounding_box"}
+{"content": "last_char_y = bbox[1]", "url": "https://forum.cogsci.nl", "title": "right coordinate of the bounding box"}
+{"content": "```", "url": "https://forum.cogsci.nl", "title": "top coordinate of the bounding box"}
+{"content": "To calculate the width of a Text element:\n\n1. Create a Canvas object.\n3. Calculate the width of a text (without rendering it) using the `text_size()` function.\n\n```python\nmy_text = 'sigmundai'\nmy_canvas = Canvas()\nwidth, height = my_canvas.text_size(my_text)\n```", "url": "https://forum.cogsci.nl", "title": "How to calculate the width of a Text element"}
+{"content": "To adjust the position of a Text element based on its length:\n\n1. Create a Text element on your canvas with the desired sentence.\n2. Use the `boundingbox` property of the Text element to get the bounding box of the entire text string.\n3. Calculate the width of the Text element by subtracting the `left` value from the `right` value of the bounding box.\n4. Set the `x` property of the Text element to the negative half of the text width to center the text horizontally on the canvas.\n5. If needed, you can also adjust the `y` property of the Text element to vertically position the text as desired.\n\n```python\nmy_text = 'sigmundai'\nmy_canvas = Canvas()\nmy_canvas['sentence'] = Text(my_text[:-1], center=False)\nbbox = my_canvas['sentence'].rect", "url": "https://forum.cogsci.nl", "title": "How to adjust the position of a Text element based on its length"}
+{"content": "text_width = bbox[2] - bbox[0]", "url": "https://forum.cogsci.nl", "title": "important: not bounding_box"}
+{"content": "my_canvas['sentence'].x = -text_width", "url": "https://forum.cogsci.nl", "title": "right - left"}
+{"content": "my_canvas['sentence'].y = 0", "url": "https://forum.cogsci.nl", "title": "move the text left so that it runs until the center of the screen"}
+{"content": "```", "url": "https://forum.cogsci.nl", "title": "set the vertical position as needed"}
+{"content": "To use a response collected in a KEYBOARD_RESPONSE item in a run-if expression to select which item is run:\n\n1. In a SEQUENCE item, set the run-if expression of one child item to `response == 'left'` to execute it only if the left arrow was pressed.  \n2. Set the run-if statement of another child item to `response == 'right'` to execute it only if the right arrow was pressed.\n3. Run-if expressions should be valid Python expressions.", "url": "https://forum.cogsci.nl", "title": "How to use a response in a run-if expression"}
+{"content": "To use JavaScript code in an experiment running in OSWeb:\n\n1. Insert an INLINE_JAVASCRIPT item into your experiment\n2. For code that needs to be executed during preparation (for example the preparation of a trial in the case of a trial SEQUENCE), use the 'Prepare' phase\n3. For code that needs to be executed during executing, use the 'Run' phase\n4. Avoid naming conflicts with variables defined in the other parts of the experiment\n5. Global variables from INLINE_JAVASCRIPT items are also available in the user interface, for example in run-if expressions\n6. Variables from the user interface, for example response variables, are available as global variables in INLINE_JAVASCRIPT items", "url": "https://forum.cogsci.nl", "title": "How to use JavaScript code in an OSWeb experiment"}
+{"content": "To create a recall trial where an image is shown and the participant types in a response in OSWeb, you can use an INLINE_HTML item with the following code:\n\n\n```html\n<input type='text' name='text_response'>\n<img id='my_image_element'>\n<input type='submit' value='ok'>\n\n<script>\n// Assume that the variable `my_image` defines the name of the image in the file pool\ndocument.getElementById('my_image_element').src = pool[my_image].data.src\n</script>\n```", "url": "https://forum.cogsci.nl", "title": "How to show a form with image and text input in OSWeb"}
+{"content": "To run an experiment in a web browser using OSWeb, follow these steps:\n\n1. Open the Experiment Properties and select 'In a browser with OSWeb (osweb)' in the 'Run experiment' section.\n2. Click any of the 'Run' buttons to start the experiment.\n3. If the experiment is not compatible with OSWeb, an error message will appear that details the compatibility issues. \n4. If there are no compatibility issues, the experiment will open in a new browser window. Note that even though the experiment is running in a web browser, it is still executing locally on your own computer.\n5. When the experiment is finished, the data will be downloaded in `.json` format. This data file can then be converted to `.xlsx` or `.csv` format for further analysis.", "url": "https://forum.cogsci.nl", "title": "How to run an experiment in a web browser using OSWeb"}
+{"content": "For more control over OSWeb experiments, you can access the OSWeb and JATOS control panel:\n\n1. Click on the Tools menu\n2. Select 'OSWeb and JATOS control panel'\n3. The OSWeb and JATOS control panel will appear, offering configuration options such as: \n   - Possible subject numbers when running in JATOS\n   - Enabling fullscreen mode \n   - Showing the OSWeb welcome screen\n   - Bypassing compatibility checks\n   - Customizing welcome text\n   - Specifying external JavaScript libraries to load", "url": "https://forum.cogsci.nl", "title": "How to access the OSWeb control panel"}
+{"content": "To host an experiment online so that it can be run by participants, you need to publish it to JATOS:\n\n1. In OpenSesame, open your experiment and check that it is compatible with OSWeb \n2. In the OSWeb and JATOS control panel, configure your JATOS server (such as https://jatos.mindprobe.eu) and your API token, which you can generate by logging into your JATOS server in a web browser.\n3. In the OpenSesame menu, select 'Save and publish to JATOS' (or 'Save and publish to MindProbe' depending on which server you have configured).\n4. Your experiment will now be uploaded to your JATOS server.\n9. Once uploaded, you can generate links by logging into your JATOS server in a web browser to allow participants to run your experiment in a web browser\n\nAlternatively, you can export the experiment to a JATOS archive (jzip) using the OSWeb and JATOS control panel, and import this archive through the JATOS server.", "url": "https://forum.cogsci.nl", "title": "How to host an experiment online using JATOS / MindProbe"}
+{"content": "2. In the OSWeb and JATOS control panel, configure your JATOS server (such as https://jatos.mindprobe.eu) and your API token, which you can generate by logging into your JATOS server in a web browser.\n3. In the OpenSesame menu, select 'Open from JATOS' (or 'Open from MindProbe' depending on which server you have configured).\n4. Select an experiment from the list that appears.\n5. The experiment will now be downloaded from your JATOS server.\n6. Once downloaded, the experiment will be opened in OpenSesame.\n\nAlternatively, you can export the experiment from the JATOS server, and import it as a JATOS archive (jzip) using the OSWeb and JATOS control panel.", "url": "https://forum.cogsci.nl", "title": "How to download an experiment from JATOS / MindProbe"}
+{"content": "To play a video file for a randomly selected duration in OpenSesame:\n\n1. In an INLINE_SCRIPT item, create a list of possible durations, e.g., `durations = [300, 350, 400, 450, 500]`\n2. Use `random.choice(durations)` to randomly select a duration from the list and store it in a variable, e.g., `video_duration = random.choice(durations)`\n3. In the MEDIA_PLAYER_MPY item, set the duration field to `{video_duration}`", "url": "https://forum.cogsci.nl", "title": "How to play a video file for a random duration in OpenSesame"}
+{"content": "To optimize video playback performance in OpenSesame:\n\n1. Ensure that your video files are in a format that is well-supported by the video backend you are using (e.g., `.mp4` for the `mpy` backend)\n2. Compress your video files to reduce their file size while maintaining acceptable quality, using tools like FFmpeg or Handbrake\n3. Use a reasonable frame rate for your videos (e.g., 30 fps) to balance smoothness and performance\n4. If possible, use a smaller video resolution to reduce the processing load on the computer", "url": "https://forum.cogsci.nl", "title": "How to optimize video playback performance in OpenSesame"}
+{"content": "To randomly select images and videos in OpenSesame:\n\n1. In an INLINE_SCRIPT item, create separate lists for your image and video files, e.g., `images = [\"image1.png\", \"image2.png\", \"image3.png\"]` and `videos = [\"video1.mp4\", \"video2.mp4\", \"video3.mp4\"]`\n2. Use `random.choice(images)` to randomly select an image file from the `images` list and store it in a variable, e.g., `selected_image = random.choice(images)`\n3. Similarly, use `random.choice(videos)` to randomly select a video file from the `videos` list and store it in a variable, e.g., `selected_video = random.choice(videos)`\n4. In a SKETCHPAD item, set the image source to the `{selected_image}` variable to display the randomly chosen image\n5. In the MEDIA_PLAYER_MPY item, set the video source to the `{selected_video}` variable to play the randomly chosen video", "url": "https://forum.cogsci.nl", "title": "How to randomly select images and videos in OpenSesame"}
+{"content": "To examine the data collected by an online experiment hosted through JATOS/ MindProbe:\n\n1. Log in to the JATOS server where the study is hosted, for example MindProbe.eu\n2. Navigate to the study and open the Study results page\n3. Click on 'Export Results' and select the file format (e.g. JATOS Results Archive or Data only)\n4. In OpenSesame, open the OSWeb and JATOS control panel, and use the convert OSWeb results to .csv/ .xlsx option\n4. Open the converted .csv / .xlsx file in a spreadsheet or text editor \n5. Each row corresponds to one variable logging operation, which is typically one trial (although that depends on the structure of your experiment)\n6. Check that all expected variables are present and have sensible values\n7. Look for missing values, unexpected repetitions, or other anomalies\n8. If data appears to be missing or incorrect, proceed to further troubleshooting", "url": "https://forum.cogsci.nl", "title": "How to examine the data after running an experiment online"}
+{"content": "If an experiment hosted online through JATOS/ MindProbe is not correctly logging data, you can troubleshoot by:\n\n1. Running the experiment locally, after enabling in OSWeb by selecting 'In a browser with OSWeb' in the experiment properties, to check for error messages in the browser console (F12)  \n2. Checking that all variables are initialized with default values at the start of the experiment \n3. Verifying that variables are set correctly in each trial, for example in an INLINE_JAVASCRIPT item\n4. Confirming that a LOGGER is executed at the end of each trial to write variables to the log \n5. Checking that no JATOS or JavaScript errors occur in the browser console when the experiment is run on JATOS\n6. Testing whether data is stored correctly if the experiment is aborted or navigated away from prematurely", "url": "https://forum.cogsci.nl", "title": "How to troubleshoot data logging issues in online experiments"}