Inventory (UXPROD-785)

[UXPROD-1655] Inventory. Search call number (Core functional) - thin thread Created: 17/Apr/19  Updated: 16/Sep/20  Resolved: 16/Mar/20

Status: Closed
Project: UX Product
Components: None
Affects versions: None
Fix versions: Q1 2020
Parent: Inventory

Type: New Feature Priority: P3
Reporter: Charlotte Whitt Assignee: Charlotte Whitt
Resolution: Done Votes: 0
Labels: call_number, cap-mvp, inventory, metadatamanagement, po-mvp, q1-2020-at-risk, q4-2019-at-risk, split
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Attachments: PNG File Skärmavbild 2019-12-04 kl. 14.46.11.png     PNG File Skärmavbild 2019-12-04 kl. 14.48.42.png    
Issue links:
Blocks
is blocked by UXPROD-1626 Store Effective Call Number Prefix, C... Closed
Defines
is defined by MODINVSTOR-444 add indexes for call number searching... Closed
is defined by UIIN-858 Search. Holdings. Search option for C... Closed
is defined by UIIN-990 Search. Item. Search option for Call ... Closed
is defined by UIIN-721 Holdings record. Implement new elemen... Closed
is defined by UIIN-722 Item record. Implement new element: n... Closed
Gantt End to Start
has to be done before UXPROD-1714 FOLIO wide search by call number for ... Open
Relates
relates to UXPROD-2002 Implement Normalized Call Number for ... Closed
Epic Link: Inventory
Analysis Estimate: Small < 3 days
Analysis Estimator: Charlotte Whitt
Front End Estimate: Medium < 5 days
Front End Estimator: Niels Erik Nielsen
Front-End Confidence factor: Low
Back End Estimate: Medium < 5 days
Back End Estimator: Niels Erik Nielsen
Estimation Notes and Assumptions: Outstanding questions:
* Implemented as a search index or as a persisted field containing the normalized version of the call number?
* Hard-code normalization rules or configurable?
Development Team: Prokopovych
PO Rank: 125
PO Ranking Note: CW: Aligned PO rank with Calculated Total rank
Rank: Chalmers (Impl Aut 2019): R2
Rank: Chicago (MVP Sum 2020): R1
Rank: Cornell (Full Sum 2021): R1
Rank: Duke (Full Sum 2021): R1
Rank: 5Colleges (Full Jul 2021): R1
Rank: FLO (MVP Sum 2020): R1
Rank: GBV (MVP Sum 2020): R2
Rank: hbz (TBD): R1
Rank: Hungary (MVP End 2020): R1
Rank: Lehigh (MVP Summer 2020): R1
Rank: MO State (MVP June 2020): R1
Rank: TAMU (MVP Jan 2021): R1
Rank: U of AL (MVP Oct 2020): R1

 Description   

This feature capture the work planned for the Core functional team

The elements Call number, and Call number type are implemented in the Holdings and Item records as of Q4 2018.

Use cases:
Multiple times daily catalogers and other library staff need to search by call number; system needs to search normalized version, not exact text string.

Several libraries have expressed a requirement on being able to search by normalized call number - besides search on call number as an exact string, we'd also need search where we handle removing of misc. qualifiers, and more.

Summary of requirements discussed with SMEs:
So this would be the best solution for the searching of call numbers:

  1. String/phrase index for prefix and call number (no keyword index)
  2. Searching must be possible without spaces and/or special characters
  3. Additionally: exact search (with spaces and special characters)
    • could be customized (per tenant) with a selection list of special characters to be taken into account in a search.
  4. No automatically truncation
    • Wildcards
    • Wildcard for explicitely right truncation, e.g. “*”
    • Wildcard for exact one character, e.g. “!”
  5. Call numbers
    • geo 11
    • geo 12
    • geo 100
    • geo 123
  6. Search variants
    • geo 1! → geo 11 and geo 12
    • geo 1!! → geo 100 and geo 123
    • geo 1* → geo 11 and geo 12 and geo 100 and geo 123
  7. Wildcard for 0-n characters, e.g. “#”
  8. Character classes
    • germ 3[567]0* = germ 350* | germ 360* | germ 370*
  9. Boolean operators, also to combine different search criteria
    • AND
    • OR
    • NOT

Documentation:

  • - - - - - - - - - - - - - - - - - - - - - - -

Examples:
tit=letters per=goethe,*
(Title must contain “letters”, author is Goethe,* ‑ standard operator is AND)
per=bach,j* NOT sgn=mus*
(Author is bach,j*, call number does not start with “mus”)
tit cookbook AND (sgn=math* OR sgn=comp*)
(the cookbook series from O’Reilly)

Some more complex examples from GBV libraries *before *normalization (added by Felix Hemme):

  • 8 G.B.439 :6
  • JUR:R III:54:(1):Schm:1850
  • 94 NF 14/1:3792-3835
  • 1990/146 4°
  • 426/083 4° SH 34
  • Z 557: 54.1961/62,7-12
  • Y 43839 (2017/18)


 Comments   
Comment by Erin Nettifee [ 22/Apr/19 ]

Yes, this is a requirement for Duke (go-live). We'd also want the call number to display on the search results screen (which we assume is the inventory pane sorted down.)

We also want to be clear about the specific issue of searching by normalized call number - staff need to be able to search by the call number that they see on the book, or on the web catalog entry, or from a bib record. In other words, if normalization is changing the call number formatting, staff can still search by the call number before it is normalized to have it return the result as expected.

We would be glad to discuss this need further.

Comment by Felix Hemme [ 13/May/19 ]

Some more complex examples from GBV libraries before normalization:

  • 8 G.B.439 :6
  • JUR:R III:54:(1):Schm:1850
  • 94 NF 14/1:3792-3835
  • 1990/146 4°
  • 426/083 4° SH 34
  • Z 557: 54.1961/62,7-12
  • Y 43839 (2017/18)
Comment by Felix Hemme [ 14/May/19 ]

Outcome of a conversation with Uschi Klute:
It will be necessary to remove one or the other character for creating an index entry. Therefore, only one search option will probably not accommodate all FOLIO libraries. It would be optimal to be able to define at least per tenant the characters themselves which are ignored during the search. For some the slash is elementary, for others the plus sign. And for others, both should be ignored. This is an example for multiple indices for the call no "Z 557: 54.1961/62,7-12" from our current ILS:
Call number with blanks, dot, comma, dash: z 557: 54.1961/62,7-12 OR z 557 54.1961 62,7-12
Call number without blanks and special characters: z55754196162712

Comment by lew235 [ 06/Jun/19 ]

There is a need for normalized call numbers for reporting as well.
https://folio-org.atlassian.net/wiki/display/RPT/Shelf+List+Location+Report+Prototype

Comment by Charlotte Whitt [ 06/Jun/19 ]

Yes lew235, that's correct. I can add 'Reporting' in the description.

Comment by Cate Boerema (Inactive) [ 08/Aug/19 ]

Marking this Open, as Draft suggests that stories are actively being written.

Comment by Cate Boerema (Inactive) [ 23/Sep/19 ]

Charlotte Whitt, lew235 et al. Do we have a decision on whether call number searching should happen on the "normalized" version as stated in the story description or on the human-readable version as suggested in the comments here from Erin Nettifee and, if I'm understanding correctly, Felix Hemme? Or maybe we should have a combined search index which allows searching both at once?

Comment by lew235 [ 23/Sep/19 ]

Based on our discussion in the call number notes document about the definition of normalized, I think we want the search to be on the human readable call number. But we do need to be able, as Felix notes above, to define search behavior around spaces and punctuation. If we need to search the normalized call number in order to accomplish this, then I think we would need both at once.

Comment by Marc Johnson [ 22/Oct/19 ]

Charlotte Whitt Cate Boerema Is the intention that in this feature it will be possible to sort the instance search results by the sortable normalised call number (of items)?

Comment by Cate Boerema (Inactive) [ 22/Oct/19 ]

Based on our discussion in the call number notes document about the definition of normalized, I think we want the search to be on the human readable call number.

Thanks lew235, what elements of the human readable call number? Options: <EffectiveCallNumberPrefix> <EffectiveCallNumber> <EffectiveCallNumberSuffix> <Volume> <Enumeration> <Chronology> <EffectiveCopy>

But we do need to be able, as Felix notes above, to define search behavior around spaces and punctuation. If we need to search the normalized call number in order to accomplish this, then I think we would need both at once.

It's a bit confusing because the term "normalized call number" seems to be used to refer to two different things: (1) the sortable call number (which will be developed using Tod's code) and (2) the searchable call number (which, as I understand it, is the human-readable call number and can be searched in ways that, for example, ignore punctuation). I would be surprised if #1 is equal to #2. I think we should assume for now that they are not.

Comment by Cate Boerema (Inactive) [ 22/Oct/19 ]

I just reviewed the discussion on Slack https://folio-project.slack.com/archives/GMAPRTZ6V/p1569232347019600 and I am still not sure where we landed on this Tod had the last comment and he suggested we search on <EffectiveCallNumber>.

I think we might want to call a meeting to discuss call number searching with the sub-group and include Marc Johnson and maybe a frontend developer, as well. I see there is tons of good info in the Notes document (this section: https://docs.google.com/document/d/1FMl-_oNR6k-wVDQrZeMT_V9-ZDDaZBzdiipx0OQii9E/edit#heading=h.6cxl044jh93x).

Comment by Cate Boerema (Inactive) [ 25/Oct/19 ]

Given we only have 2.5 sprints left and this isn't started (or dev-ready) I think we should label it as at-risk

Comment by Jakub Skoczen [ 02/Dec/19 ]

Cate Boerema Marc Johnson

Charlotte Whitt and I discussed this ticket and concluded that while this is similar to UIIN-647 Closed there doesn't seem to be a simple. backend-only solution here.

The reason is that a search for a call number like "a bc" may mean "a callnumber bc with a prefix a" or "a call number 'a bc'". This ambiguity requires more advanced normalization in the backend OR a combination of a front-end and backend solution.

Comment by Jakub Skoczen [ 02/Dec/19 ]

Marc Johnson Julian Ladisch Adam Dickmeiss

One potentially solution could be:

1. add a new normalisation function in RMB called stripPunctuation. This function would remove all punctuation and whitespace from a given field before creating an index, e.g a callNumber like "a/b/c 123-3" would result in "abc1233". When enabled for a field, this function would also apply during search.

2. Have the UI expand the search into an OR expression across three fields: callNumberPrefix, callNumber, callNumberSuffix like so:

given a search for "a bc d" the UI would generate a query callNumberPrefix=a AND callNumber=bc AND callNumberSuffix=d OR callNumberPrefix=a AND callNumber="bc d" OR callNumber="a bc" and callNumberSuffix=d OR callNumber="a bc d"

Ugliness of the solution noted. Ideas for something less ugly welcome.

Comment by Marc Johnson [ 03/Dec/19 ]

Jakub Skoczen

Have the UI expand the search into an OR expression across three fields: callNumberPrefix, callNumber, callNumberSuffix like so:

given a search for "a bc d" the UI would generate a query callNumberPrefix=a AND callNumber=bc AND callNumberSuffix=d OR callNumberPrefix=a AND callNumber="bc d" OR callNumber="a bc" and callNumberSuffix=d OR callNumber="a bc d"

My understanding is that we are trying to move away from this kind of query construction in the UI and the goal of this feature is to provide a simple (from a clients perspective) way of doing this.

Comment by Charlotte Whitt [ 03/Dec/19 ]

Marc Johnson - please see a more complete list of misc. call number searches listed in UIIN-857 Closed (umbrella issue)

Comment by Marc Johnson [ 03/Dec/19 ]

Charlotte Whitt

Thanks for the reference. I'm not actively investigating this issue, as I thought that Jakub Skoczen was exploring a solution with Adam Dickmeiss within the core platform team.

Comment by Charlotte Whitt [ 03/Dec/19 ]

Hi Jakub Skoczen - lew235 and I just talked about your proposal, and also discussed your suggestion using [spaces] between call number prefix, call number, and call number suffix, like in your example: given a search for "a bc d".

The problem is that e.g. the examples in UIIN-857 Closed , there are spaces in the call number itself - e.g.:

  • H Germ 350/35: 1
  • H Germ 350/35: 2
  • F Germ 350/1

Then H is a call number prefix in

  • H Germ 350/35: 1
  • H Germ 350/35: 2

And F is a call number prefix in

  • F Germ 350/1

But there is also two expected spaces in the call number e.g. Germ 350/35: 1

So Laura and I was talking about if we in the search could use e.g. | [pipe] or _ [under score] as delimiter to indicate what is call number prefix, what is call number, what is call number suffix, e.g. H!Germ 350/35: 2, where H is the prefix, and Germ 350/35: 2 is the call number.

And then what to do if we only have a call number and a call number suffix (which is typical use for Chalmers, as I have understood it). Maybe: |Germ 350/35: 1|Z (a made up example)?

Comment by Tod Olson [ 04/Dec/19 ]

What kind of search are we talking about, an ordered browse or something closer to a keyword result set? I can't quite tell from the comments so far, and it's important.

From the examples, it seems like the US libraries coming from Aleph and OLE expect an ordered browse, but the GBV libraries coming from LBS expect something more like a string search or relevance ranking. Is there agreement on the kind of result set?

If we are talking about an ordered browse search, or even if there is a need to sort by call number, then there's a real need for a sort key. And the sort key algorithm might not be the same for all sites.

But back to the inventory call number search, what kind of result set is expected? It's not entirely clear from what I have read.

Comment by Charlotte Whitt [ 04/Dec/19 ]

Hi Tod Olson - we are planning to implement following search options:

Call number, keyword

  • Call number, eye readable
  • Call number, normalized

Search for Call number are to be implemented as a search option in the drop down menu in both the Holdings segment and the Item segment:

Search options Holdings record Search options Item record

See list of examples on call number search: UIIN-857 Closed .

And story for implementation: UIIN-858 Closed .

Comment by Cate Boerema (Inactive) [ 10/Dec/19 ]

Hi Tod Olson call number sort is covered in a separate feature: UXPROD-2002 Closed

Comment by Cate Boerema (Inactive) [ 30/Jan/20 ]

This is blocked awaiting the Core Platform work. Marking blocked

Comment by Cate Boerema (Inactive) [ 28/Feb/20 ]

Hi Charlotte Whitt I think a couple of the stories defining this feature don't belong here anymore ( UIIN-857 Closed and UIIN-990 Closed ). Could you please split those off into another feature for Q2 and rename this one so that it's clear what we decided to do for Q1? That will help us better track when this feature is complete. Thanks!

Comment by Cate Boerema (Inactive) [ 02/Mar/20 ]

Hi Charlotte Whitt I think, if we remove the two stories that are out of scope for Q1, this feature could be moved into In review.

Comment by Cate Boerema (Inactive) [ 16/Mar/20 ]

Charlotte Whitt can you please remove the two extraneous stories and mark this closed?

Comment by Charlotte Whitt [ 16/Mar/20 ]

Okay . Done.

Generated at Fri Feb 09 00:17:22 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.