View source for Help talk:CirrusSearch

{{ambox|text=This talk page is about the usage of CirrusSearch. See [[Extension:CirrusSearch|installation]], or [//phabricator.wikimedia.org/tag/cirrussearch development], or [[BUGREPORT|bugs]], or [[w:Help talk:Searching|Wikipedia]] for related discussions.}}
{{Archives|list=
* [[/Flow|Original Flow board]]
* [[/2014|2014]]
* [[/2015|2015]]
* [[/2016|2016]]
* [[/2017|2017]]
* [[/2018|2018]]
* [[/2019|2019]]
* [[/2020|2020]]
* [[/2021|2021]]
* [[/2022|2022]]
* [[/2023|2023]]
* [[/2024|2024]]
* [[Special:PageHistory/Help talk:CirrusSearch/Hidden|hidden topics]]
See also [[/en]].
}}

== Does boost force sort=relevance? ==

If I am using a defined sort order, such as, say, <code>&sort=last_edit_desc</code>, what happens if I choose a bounded or boosted search on, say, <code>100km,San Francisco</code> ? Does it turn off my sort selection, and switch it to <code>relevance</code> instead? If not, what is the meaning of a search which contains both of those (or any sort option other than relevance) ? <small>(subscribed)</small> [[User:Mathglot|Mathglot]] ([[User talk:Mathglot|talk]]) 05:47, 19 December 2024 (UTC)

:The sort param will take precedence over what's inside the query.
:All the search keywords that do impact relevance only like:
:* prefer-recent
:* boost-template
:* boost-neartitle
:* morelike
:have no impact when sort is not relevance. To be precise morelike may still have an impact but it should be very marginal and there's not much value in using it when sort is not relevance.
:Other keywords (including bounded geo searches like neartitle/nearcoord) will have an impact on the set of documents returned regardless of the sort parameter.
:We could possibly emit a warning when such keywords are used in conjunction with a sort param that is not relevance. [[User:DCausse (WMF)|DCausse (WMF)]] ([[User talk:DCausse (WMF)|talk]]) 09:07, 22 September 2025 (UTC)
::[[User:DCausse (WMF)|DCausse]], thanks. Note that I have added new section [[Help:CirrusSearch#Interaction among search options|§&nbsp;Interaction among search options]] to the page, based on my possibly faulty understanding of what you wrote above. Please correct it as needed, and please expand the section to include any other interactions among search options that users should know about. Thanks in advance, [[User:Mathglot|Mathglot]] ([[User talk:Mathglot|talk]]) 19:54, 22 September 2025 (UTC)

== How to search the fields of the File information template on Commons? ==

Nearly all of the over 110 million files on Commons use this standardized template that specifies various useful metadata like date taken and file description. How to make use of this data and search it?

For example, how could one search file ''description'' for a term like "Kathmandu" (as asked about by another user) like on can search with <code>intitle</code>. <code>description:"Kathmandu"</code> does show some results but I don't know what it does and the results don't have that word in their description. I could not find info on this at [[mw:Help:CirrusSearch]] either. Info how to search specified fields of [[c:Template:Information]] should be added here.

* One could also use this to infer categories (such as by reading the date field and then adding it to a category by date like "[[:c:Category:Videos by year|Videos of {year}]]") as proposed [[c:Commons:Bots/Work_requests/Archive_18#Auto-addition_of_inferrable_categories|here]].
* For example, I found that many files in <code>[https://commons.wikimedia.org/w/index.php?title=Special:Search&limit=500&offset=0&ns6=1&search=deepcategory%3A%22NASA+videos+from+unidentified+year%22+deepcategory%3A%22Videos+of+2020%E2%80%8E%22+filetype%3Avideo&advancedSearch-current={%22fields%22:{%22filetype%22:%22video%22}} deepcategory:"NASA videos from unidentified year" deepcategory:"Videos of 2020‎"]</code> have been miscategorized into Videos of 2020 (and thus should not be copied into "NASA videos in 2020" from there) where they have the correct date in the date field which is why I'd like to use that to correct that as well as copy them to their year category in [[:c:Category:Videos from NASA by year]].
* This may also be needed for a date range filter, see [[phab:T329961]]. I'd like to search the date field but there is no information on how to do that at [[Help:Searching]] but I think it's already possible if I remember correctly.
* Another example: one could set subcats of [[:c:Category:Media from scholarly journals]] depending on the link in the source field. For example, files with an URL starting with <code><nowiki>https://www.nature.com/</nowiki></code> or a DOI that resolves to one should be in a respective subcat of [[:c:Category:Media from Nature Publishing Group journals]].
* One could also search the source field to put files into cats like [[:c:Category:Audio files from Soundcloud.com]] and so on.
* Also how can one search for files from a specific uploader? (I'd like to check which of my video2commons uploads [[c:Commons_talk:Video2commons#Suddenly_imports_at_low_quality|were imported below resolution at source]].)
EBernhardson (WMF) said {{tq|Unfortunately, the image description is simply an argument to a template. CirrusSearch doesn't do anything at that level and can't be that specific.}}.
I think the '''best workaround currently would be to use the ''insource'' search operator''' with the field name first so for example I searched for <code><nowiki>insource:"|source=[https://soundcloud.com</nowiki></code> to identify files for [[:c:Category:Audio files from Soundcloud.com]]. I think easily searching fields of the File pages' Information template could be enabled by
# Developing some '''regex''' that searches for any content after e.g. <code>|source=</code>
# Creating some '''alias''' for it so instead of writing some complex regex query every time one can simply enter e.g. <code>info-source:"soundcloud.com"</code>
Please comment what you think about this proposed way to make this possible and if you have any info on what would be needed for that. Would be great if somebody could develop such (a) regex(es) if there is no better way to search specific fields of the Information template. It's great that files have that structured metadata but it could be much more useful if it was searchable.

[[Commons:Village_pump/Technical/Archive/2024/10#How_to_search_fields_of_files'_Information_template?|Previously asked here]]. Maybe [[c:Module:Information]] could be used for this somehow. [[User:Prototyperspective|Prototyperspective]] ([[User talk:Prototyperspective|talk]]) 16:39, 5 March 2025 (UTC)

== Regex search speed ==

In my experience bare regex searches seem to work even without any other terms. For example https://syl.wikipedia.org/w/index.php?go=Go&search=insource%3A%2F%5C%7B%5C%7BINTERWIKI%2F&title=%EA%A0%9B%EA%A0%A4%EA%A0%A1%EA%A0%A6%EA%A0%A1%3ASearch&ns0=1 completes quickly and returns 43 results. Why is that, and can the warning be removed? [[User:Pppery|* Pppery *]] [[User talk:Pppery|<sub style="color:#800000">it has begun</sub>]] 14:24, 1 April 2025 (UTC)

:The regex search is from my understanding still searching the whole database unless the search area is narrowed with an index based parameter or filter. The database for syl-wiki will just be not very large so that the regex search is faster than the timeout. However, the help page is for every possible Mediawiki installation, so any potential caveat has to be addressed. —<span style="white-space:nowrap"> [[User:Speravir|Speravir]] ([[User talk:Speravir|talk]]) – 23:47, 11 April 2025 (UTC)</span>
:: Postscript, see [[phab:T411112]] which corrected this longstanding misconception. [[User:Pppery|* Pppery *]] [[User talk:Pppery|<sub style="color:#800000">it has begun</sub>]] 16:35, 4 December 2025 (UTC)

== not displayable chars  U+10FFFF  ==

In chapter  "Substitutions for some metacharacters"  columns 'CirrusSearch' and text ,  all chars explained as  "􏿿" is U+10FFFF" are displayed by a default pavement char as for not existing chars.Adapt or useless ?  Thanks. -- [[User:Wladek92|Christian 🇫🇷 FR 🚨]] ([[User talk:Wladek92|talk]]) 08:04, 7 April 2025 (UTC)
:You may see a “default pavement char”, I see a glyph which has the unicode number (very small) imprinted. In general, you get a glyph of a font selected by your browser, if there is one; otherwise the behaviuor apparently depends on the browser. At this very place it means, there is actually the character U+10FFFF visible and you can execute a copy and paste action. —<span style="white-space:nowrap"> [[User:Speravir|Speravir]] ([[User talk:Speravir|talk]]) – 00:05, 12 April 2025 (UTC)</span>

== Explicit sort by pagename ==

I would have thought that ordering search results by PAGENAME would have been one of the most typical sort options, but I don't see it listed. I tried various searches to come out alphabetical by PAGENAME but was not able to. Surely pagename must be an index item, and this couldn't be that hard; I would have thought it would be one of the top choices, even the default sort option (for non-huge result sets, anyway). Any chance it could be added? [[User:Mathglot|Mathglot]] ([[User talk:Mathglot|talk]]) 21:29, 12 November 2025 (UTC)

:At scale, nothing is ever simple. However there has been some recent movement for this after a few blockers were unblocked and it seems they are about to evaluate if it would be possible to create this option. Status is tracked at [[phab:T40403]]. —[[User:TheDJ|Th<span style="color: green">e</span>DJ (Not WMF)]] ([[User talk:TheDJ|talk]] • [[Special:Contributions/TheDJ|contribs]]) 22:47, 12 November 2025 (UTC)

:At the moment, it works with a user script, see [[User:PerfektesChaos/js/resultListSort]] or, if you understand German, [[:de:Benutzer:PerfektesChaos/js/resultListSort]]. —<span style="white-space:nowrap"> [[User:Speravir|Speravir]] ([[User talk:Speravir|talk]]) – 01:47, 21 November 2025 (UTC)</span>

==Discussion at [[w:Wikipedia:Village pump (technical)#Early Explorations Into Semantic Search: Phase 0]]==
[[File:Symbol watching blue lashes high contrast.svg|25px|link=|alt=]]&nbsp;You are invited to join the discussion at [[w:Wikipedia:Village pump (technical)#Early Explorations Into Semantic Search: Phase 0]].  <span style="border:3px outset;border-radius:8pt 0;padding:1px 5px;background:linear-gradient(6rad,#86c,#2b9)">[[User:Sdkb-WMF|<span style="color:#FFF;text-decoration:inherit;">Sdkb-WMF</span>]]</span> <sup>[[User talk:Sdkb-WMF|'''talk''']]</sup> 19:51, 11 January 2026 (UTC)<!-- [[Template:Please see]] -->

== Logical OR (how to use?) & bug ==

I may have overread how to do this in the docs. I would just ask here if this wish was implemented: [[m:Community Wishlist/W442]]. How to use it – e.g. ''intitle:this OR intitle:that'' should show pages with either this or that in the title.

Moreover, why is this wish not found when [https://meta.wikimedia.org/w/index.php?search=intitle%3AChart+incategory%3A%22Community+Wishlist%2FWishes%22&title=Special%3ASearch&profile=advanced&fulltext=1&ns0=1 searching for ''intitle:Chart'']: [[m:Community Wishlist/W495]]?<br> [[User:Prototyperspective|Prototyperspective]] ([[User talk:Prototyperspective|talk]]) 11:45, 3 February 2026 (UTC)

==Searching using "insource:" is not working as expected in Wikisource==
I'd like to be able to search for misspellings in Wikisource which have not already been identified as original to the source.

In English Wikipedia I can search <code>"an unique" -insource:"not a typo an"</code> and I get uses of "an unique" which have not been marked, but in Wikisource if I search <code>"pronounciation" -insource:"SIC pronounciation pronunciation"</code> it does not eliminate the archaic spellings I have marked as <code><nowiki>{{SIC|pronounciation|pronunciation}}</nowiki></code> in, for example, [[s:Page:The American Democrat, James Fenimore Cooper, 1838.djvu/125]].

What's going on or what am I doing wrong? Thank you.[[User:SchreiberBike|SchreiberBike]] ([[User talk:SchreiberBike|talk]]) 14:45, 6 February 2026 (UTC)

:The problem seems to be that you are not searching in the Page namespace, and pages that are transcluded into the Main namespace seem to have their text indexed in that namespace, but not their underlying wikitext.
:Your current query searches for <code>"pronunciation"</code> in text and '''NOT''' <code>"SIC pronounciation pronunciation"</code> in wikitext. Since the wikitext isn't indexed, your results are off.
:More evidence that it works this way (I'm inferring the details; I'm not familiar with the indexing set up for English Wikisource):
:* If you search for just <code>insource:"SIC pronounciation pronunciation"</code> you only get one result, even though we know there are more. If you add the Page namespace you get 29 results.
:* If you change your query to <code>insource:"pronounciation" -insource:"SIC pronounciation pronunciation"</code> you only get one result, but it is correct, because all of its text is in the Main namespace, not transcluded.
:So, if you are doing an <code>insource</code> on English Wikisource, you should add the Page namespace.
:I can see why the Page namespace isn't included in searches by default, since you'd get duplicate results for regular searches—the Page namespace version and the Main namespace version. And the wikitext isn't indexed twice because it's an expensive index that isn't used nearly as often. Unfortunately those facts collide in your query to give unexpected results. [[User:TJones (WMF)|TJones (WMF)]] ([[User talk:TJones (WMF)|talk]]) 16:12, 6 February 2026 (UTC)
::The lack of <code>insource</code> results in mainspace is not optimization. It’s a feature: <code>insource</code> searches whatever one sees in the edit box – so you can use it to search for template transclusion syntax (or, in case of Wiktionary, <code>&lt;pages&gt;</code>) rather than whatever is transcluded. —[[User:Tacsipacsi|Tacsipacsi]] ([[User talk:Tacsipacsi|talk]]) 21:17, 6 February 2026 (UTC)
::Thank you. It took me a bit, but I found if I put <code>Page:</code> before the search string, that means I can search that namespace. So much I don't know, but there are so many helpful people. Keep up the good work. [[User:SchreiberBike|SchreiberBike]] ([[User talk:SchreiberBike|talk]]) 22:12, 6 February 2026 (UTC)
:::[[File:Advanced Search Widget Adding Page Namespace to Search.png|thumb|300px]]
:::<code>Page:</code> is good if you only want to search the Page namespace, but you miss the result for ''The Esperanto Teacher'' that is in the Main namespace. You can search both the Main namespace (along with all the default namespaces) and the Page namespace at the same time.
:::If you have the default Advanced Search widget enabled, click "Search In" then scroll and look for Page or type it in the search box.
:::[[File:Add Page Namespace to Search in the Old UI.png|thumb|300px]]
:::If you have the Advanced Search widget disabled, then in the old UI click on Advanced, then click on the Page namespace. [[User:TJones (WMF)|TJones (WMF)]] ([[User talk:TJones (WMF)|talk]]) 16:13, 7 February 2026 (UTC)

== Go Button ==

Where the "Go" button is supposed to be? Maybe just "Search". Or maybe in a MediaWiki skin unknown to me.

I was trying to understand if the page is up to date, so to translate it. Thanks :) [[User:Valerio Bozzolan|Valerio Bozzolan]] ([[User talk:Valerio Bozzolan|talk]]) 09:44, 14 April 2026 (UTC)

:@[[User:Valerio Bozzolan|Valerio Bozzolan]] thanks for working on the translation!
:The ''Go'' button is indeed present in older skins like monobook where below the dropdown with suggestions you can see two buttons with ''Go'' and ''Search'', see: https://en.wikipedia.org/wiki/Main_Page?useskin=monobook (search box is on the navigation panel to the left).
:Newer skins got rid of this adding a last entry in the drop-down with ''Search for pages containing XYZ'' which behaves like the monobook ''Search'' button.
:I believe that it might make sense to update the doc to reflect newer skins and (possibly) keeping a short note about the ''Go'' button mentioning that it's only visible monobook.
:* vector-2022: search bar center-top with a ''Search'' button '''go'''ing to the matching page
:* vector (legacy): search bar top-right (or left for rtl language) with a magnifying glass button '''go'''ing to the matching page
:* monobook: search bar in the navigation panel with both a ''Go'' and ''Search'' button
:* timeless: search bar center-top with a magnifying glass button '''go'''ing to the matching page
:[[User:DCausse (WMF)|DCausse (WMF)]] ([[User talk:DCausse (WMF)|talk]]) 15:03, 14 April 2026 (UTC)

== Character sets that case insensitivity affects? ==

From my experience on wikipedia, the /i at the end of a regex search works on the latin alphabet and the greek. Is there a good way to get a list of the character sets where a character can be brought into acceptable for the search results based on that? (For example, does it apply to the Hebrew Alphabet or the Geʽez script?) [[User:Naraht|Naraht]] ([[User talk:Naraht|talk]]) 17:05, 22 June 2026 (UTC)

:I'm not quite sure what you are asking, but <code>/i</code> allows additional characters to match certain characters in your regex, but sometimes there are no additional characters to match. So, <code>insource:/>>>/</code> and <code>insource:/>>>/i</code> do the same thing. For Hebrew and Ge'ez, nothing happens with <code>/i</code>, as with punctuation or numbers. Most alphabets with upper- and lowercase variants in Unicode should be affected by <code>/i</code>. I checked on the very rare lowercase Cherokee letters, and they work. <code>insource:/ᏣᎳᎩ/i</code> and <code>insource:/ꮳꮃꭹ/i</code> match the same 46 results (and catch the two unexpected instances of mixed-case "Ꮳꮃꭹ") on English Wikipedia. I expect that we are using a Unicode-aware lowercasing function that's a subset of [https://en.wikipedia.org/wiki/International_Components_for_Unicode ICU] Normalization internally. It's hard to get an exact list of what it does, but I brute forced a list in 2020 ([[User:TJones (WMF)/Notes/What the Heck Does ICU Normalization Do, Anyway?|What the Heck Does ICU Normalization Do, Anyway?]]). It has been updated since—and keeping in mind that ICU Normalization does more than upper- and lowercase normalization, which <code>/i</code> does not—but that should give you an idea if you really want a long list to look over. There are still occasionally weird corner cases like <code>insource:/ΣιΣ/i</code> matches <code>σις</code>, but <code>insource/σισ/i</code> does not. (To further complicate things, there is also a bit of normalization that happens somewhere along the chain of your browser, your operating system, the Mediawiki software, PHP, Java, and back that will do things like silently convert e + combining ´ into single character, é, even if you don't want it to—but that's a whole different can of worms) Hope that helps. — [[User:TJones (WMF)|TJones (WMF)]] ([[User talk:TJones (WMF)|talk]]) 21:33, 22 June 2026 (UTC)