semantic search for lecture info example

(misc) C:\Users\peter\code-dev-22\one-off-scripts>python run_semantic_search.py -n 10 -q "will this be on the exam?"
C:\Users\peter\code-dev-22\one-off-scripts\run_semantic_search.py:53: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\utils\tensor_new.cpp:204.)
  corpus_embeddings = torch.FloatTensor(_data)
Batches: 100%|████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 54.59it/s]
Results for query: will this be on the exam?
[{'Rank': 1,
  'Score (0-1)': 0.4468,
  'doc_dir': 'course info',
  'doc_name': 'OCR_AMS_exam_information_',
  'doc_relative_loc': 0.0,
  'doc_text': 'applied multivariate statistics exam information, spring '
              'semester 2022 fabio sigrist general information the exam is a '
              'closed book exam you may use a pocket calculator without '
              'communication abilities. the exam lasts 120 minutes. the exam '
              'consists of multiple choice questions : exam relevant is '
              'everything that was covered in the course, exercises, and the '
              'corresponding handouts. a list of topic that are excluded can '
              'be found below. programing skills in the r language will not be '
              'tested in the exam _ however ; you have to be able to know what '
              'the r functions used in the course do as well as correctly '
              'interpret r output and draw the correct conclusions from it. '
              'topics which are not exam relevant kernel pca t - sne : the '
              'formulas for the similarities in the t - sne algorithm fisher s '
              'discriminant analysis : using more than one discriminant vector '
              'decision theory for classification repeated measures : the '
              'choice between random effects or fixed effects sample questions '
              'a data point that is not an outlier in a two - dimensional '
              'subspace can be an outlier in higher dimensions. yes no? ( yes '
              ') the p - value in the chi - squared test of independence for '
              'two categorical variables is interpreted as the probability '
              'that there is no dependence. yes no? ( no ) let',
  'id_within_doc': 0},
 {'Rank': 2,
  'Score (0-1)': 0.4382,
  'doc_dir': 'lecture-transcriptions',
  'doc_name': 'SC_lecture_3_v_2_c_transcription_3_GT_failed',
  'doc_relative_loc': 97.959,
  'doc_text': 'next week, and hopefully, if we do an exam preparation towards '
              'the end of the semester, we will have a better understanding '
              'slowly, and this is how it birds. for all of us, it does. so if '
              'we conclude here, thank you very much for your attention. when '
              'seeing each other again in a week from now, nearby nobody has a '
              'good afternoon or a good evening, and until next week nobody '
              'has a degree a.',
  'id_within_doc': 48},
 {'Rank': 3,
  'Score (0-1)': 0.4267,
  'doc_dir': 'exercise-session-transcriptions',
  'doc_name': 'SC_exercise_class_1_recording',
  'doc_relative_loc': 7.692,
  'doc_text': "i'm going to go through some of this code. yes, the solutions "
              "are so ; you can submit them if you want. we don't grade for "
              "this class if, for some people, it's useful to submit because "
              'you then have to produce everything right, and it will open '
              'where you submit. it will locate it and not provide detailed '
              'feedback, but it will have a lock on whether it looks '
              'compatible with solutions. the idea is that an old student who '
              'claims credit for the course submits the exercises. for those, '
              'it will have more detailed work ; they need to be sold '
              "relatively well. if you're an old student and the parliament "
              'gives you some easy credits for this, then you need to submit '
              "them. otherwise, it's not compulsory. don't remove anything "
              'away from your exam results or anything like this. so this is '
              "not part of the grade, but you can submit it if it's good for "
              'you, right? if it makes you do more clean work on this and is '
              'the course, i think you can feel free to see it. okay, good, go '
              "outdoor one second. okay, so this is series one, right? i'm "
              "going to flick between the questions and the coat i'm going to "
              'use',
  'id_within_doc': 4},
 {'Rank': 4,
  'Score (0-1)': 0.3873,
  'doc_dir': 'lecture-transcriptions',
  'doc_name': 'SC_lecture_12_v_2_c_transcription_14_GT',
  'doc_relative_loc': 97.143,
  'doc_text': "it's just for your convenience, not required to calculate up "
              'values. sometimes, a more advanced calculator can store '
              'immediate, intermediate results and might get them back easier '
              'that sense. it might be more convenient for years until '
              'yesterday, but apart from this, the calculator does not need '
              'special statistical functions. other questions this of the '
              'module form in case further questions come to your mind later '
              "on because got information rats would come out handles or don't "
              "have any more information you know more's just to repeat. the "
              'zumadian said the stun has asked when the exam date will be '
              "fixed, and he says that it's going to be at the end of this "
              'week, and you know more than me, so to my knowledge is not yet '
              'determined. i think we will all be informed at the same time '
              'cannot have more information than you, so if there are no more '
              'questions, we will close this course now. thank you very much '
              'for your attention ; and hope you have learned something and '
              'wish you good luck with the exam. for the exam preparation, '
              'hopefully, you will all have some holiday after the exam. '
              'thanks for your attention. goodbye and take care of it.',
  'id_within_doc': 34},
 {'Rank': 5,
  'Score (0-1)': 0.3872,
  'doc_dir': 'lecture-transcriptions',
  'doc_name': 'SC_lecture_1_v_2_c_transcription_1_GT',
  'doc_relative_loc': 9.302,
  'doc_text': 'submit them as the indicated deadlines to the assistant. '
              'whether you are an old student who needs to take and examine '
              'and get it great or not, it can not know because there are six '
              'or more different programs. every program has different '
              'regulations, so you must inquire about this yourself. i just '
              'know how it works here for the maths department. here the '
              "students don't need to take exams, and then they are not "
              'required to obtain a grade. still, every program handles this '
              'differently, so in case you need to take the exam in particular '
              'in case you need to obtain a grade. your register as regular '
              'students do otherwise suggest of me the exercise us and you '
              'will see at the beginning of every chapter and have this '
              'overview, so that does not get lost in the journal of methods '
              "that we are going to see. so you see here also that's a nice "
              "sorbet if all the matches we'll cover in this course. how "
              "should i say? it's not a course where we start art today, and "
              "everything builds up. methanol method, and it's all connected. "
              'you will see a collection of different tools with different '
              "applications. they are related, obviously, but it's no building "
              'up of it. so it',
  'id_within_doc': 4},
 {'Rank': 7,
  'Score (0-1)': 0.374,
  'doc_dir': 'exercise-session-transcriptions',
  'doc_name': 'SC_exercise_class_3_recording',
  'doc_relative_loc': 34.848,
  'doc_text': "is also beneficial. it's always a good tool to start with. "
              "still, if the prince this pal components don't explain first "
              "couples don't explain a lot of the variants, then you need a "
              'tool that can recognize more complex patterns in the data, and '
              "testing is a perfect tool for doing this or so that's about. "
              "that's a question there could, so i think the question for "
              "discussion is a bit techy. ical i don't know if you had a "
              'chance to try the questions. would it be interesting for you '
              'guys for me to talk about the solution to questions for some '
              "nods and some uncertainties? it's beyond my math horizon, to be "
              'honest., only it will do it. maybe i will try what i want. '
              "it'll do. the question. i might overrun eleven a bit, but i "
              'want to encourage you to try because these things monthly care. '
              'i agree hundred percent with this. but i would not say that if '
              'you have done some level of mathematics right and know the '
              "basics of linear algebra and follow the definitions, don't "
              'think beyond your level because these are not very deep '
              "questions. very difficult. if i show you the solution now, i'll "
              'show you that the solution is not that complicated',
  'id_within_doc': 23},
 {'Rank': 8,
  'Score (0-1)': 0.3542,
  'doc_dir': 'lecture-transcriptions',
  'doc_name': 'SC_lecture_801_v_2_c_transcription_10_GT',
  'doc_relative_loc': 91.837,
  'doc_text': 'general, the second one is the ability to close book exams. for '
              "open book exams, i'm not going to die in detail. have a look at "
              "this yourself afterward, when you have more time because we're "
              'running out to littebit of time for making an example in full '
              'detail, but we have written everything that is required here in '
              'the common section as well. i suggest you have a look at this. '
              "it's beneficial for understanding how such a model work, are "
              'interpreting factors in the particular so for instance to '
              'students to hear students a just good in general and the '
              'student sixty - six here is a student that is good in close '
              'books exams and bad in open book exams instead of going round '
              'for one and score so we can nicely interpret this is one story '
              'if this was a little bit quick in case you have questions about '
              'the freedom to ask them for next time or during one of the '
              "exercise classes of its sales as well. we conclude today's "
              'lecture with a comparison on pace because at first point they '
              'might seem quite similar and years quite some similarity '
              'between sea effects and asks to there so different so the '
              'fraction mall here is given by this mole here x to data as land '
              'attempts to effect plus, you where the use of the damage the '
              'coerc',
  'id_within_doc': 45},
 {'Rank': 10,
  'Score (0-1)': 0.3434,
  'doc_dir': 'exercise-session-transcriptions',
  'doc_name': 'SC_exercise_class_2_recording',
  'doc_relative_loc': 2.817,
  'doc_text': 'going to go and i am hoping to get today in the second hour. '
              "i'm hoping to get as much participation from now guys as "
              "possible, so there are not that many of you in the room, so i'm "
              'going to ask you questions, directors, so if there are no '
              'volunteers, i will pick or person to try and answer the '
              "question. i think some people don't prefer this way, but it's "
              'good to keep you engaged and feel freyou e to share any '
              'thoughts. there are no wrong answers. there are no special '
              'questions on mathematics, right? all queues times that you ask '
              "were completely valid. then all the answers are good. so i'm "
              'first going to open the solution for series one, and so what '
              "i'in doing is i'm going to go after this, on this radio and "
              'this has been published so this has been put on line onto the '
              'meadow web page. so this is if you can find it on the mudo, '
              'right? so we have this section in which we applaud the '
              'materials related to exercise, and the solution to the serious '
              'one has been put online so that you can find this pdf. you can '
              'scroll through as i show different things and, as i mentioned, '
              'some of',
  'id_within_doc': 2},
 {'Rank': 11,
  'Score (0-1)': 0.3375,
  'doc_dir': 'exercise-session-transcriptions',
  'doc_name': 'SC_exercise_class_5_recording',
  'doc_relative_loc': 0.0,
  'doc_text': 'source : sc _ exercise _ class _ 6 _ recording _ drago _ ple _ '
              'c _ kos _ personal _ meeting _ room _ t _ 56 _ w _ zw _ 4 _ eu '
              '_ dl _ icu _ yy _ lg _ 4 _ ao _ qc _ yz _ ck _ 3 _ ie _ mk _ 0 '
              '_ transcription _ 5. txt welcome to exercise, also known as the '
              'last exercise class today. the format will be slightly '
              "different, so i'm going to talk about solutions to series five "
              "; then i was going to talk about hints for series six, and i'm "
              "planning to finish by eleven thirty with no break if that's "
              'okay. and then, if you have questions because someone asks, if '
              'you can ask questions if you have any questions regarding any '
              'of the material or anything from the course or general '
              "questions about anything else, i'm happy to answer. so i think "
              "that's the plan for it today. good. let's start with series "
              "five. some of you guys'are candidates and have done a good job. "
              "she's nice. okay, so the first question on series five was "
              'comparing a principal component and factor analysis, right? so '
              'we have spoken about this debate,',
  'id_within_doc': 0},
 {'Rank': 14,
  'Score (0-1)': 0.3316,
  'doc_dir': 'lecture-transcriptions',
  'doc_name': 'SC_lecture_3_v_2_c_transcription_3_GT',
  'doc_relative_loc': 97.959,
  'doc_text': 'seems not to be the case. i suggest you review this for next '
              'week. again, pace is something you need to review a few times '
              "to understand fully. so have a look at this. you probably don't "
              "worry if you don't understand or even look at it again. but we "
              'will have a look at it again next week, and hopefully, if we do '
              'an exam preparation towards the end of the semester, we will '
              'have a better understanding slowly, and this is how it birds. '
              'for all of us, it does. so if we conclude here, thank you very '
              'much for your attention. when seeing each other again a week '
              'from now nearby, nobody has a good afternoon or evening, and '
              'until next week nobody has a degree a.',
  'id_within_doc': 48}]
Search took 0.16 seconds