![]()
How to tune hybrid search hybrid search를 구현할 때 가장 먼저 해야할 일은 "Does hybrid actually beat its components?" 질문을 해야한다. fusion 방법인 RRF에서는 다음과 같은 수식을 사용한다: 이때 K의 의미가 중요하다:작은 K: Rank에 굉장히 민감하다. K=2 인 경우, rank1이 rank10보다 약 5.5배 강한 contribution을 갖는다. 즉, retriever는 rank 1 doc이 rank 10 doc보다 좋다는 것을 강하게 믿는다. 그래서 작은 K는 individual retriever의 top ranking을 많이 신뢰한다고 이해하는 것이 적절하다.큰 K: Rank의 차이에 둔감하다. K=61 인 경우, r..
![]()
Hybrid Search Dense retrieval은 query와 documents를 embedding vector로 만든다음, 이들의 cosine similarity를 통해서 두 texts 사이의 유사도를 계산한다: 이는 표현이 조금 달라도 의미가 비슷하면 관련된 문서를 가져올 수 있다는 장점이 있다. 예를 들어, query는 "심장이 빠르게 뛰면서 어지럽다."이고, documents는 "빈맥과 현기증이 동반될 수 있다."라면 이는 겹치는 표현은 존재하지 않지만 latent space에서 비슷한 의미를 가지고 있기 때문에 유사한 문서로 검색될 수 있다. 하지만 embedding은 문장을 하나의 representation으로 압축하기 때문에 특정 문자열이 정확히 일치한다는 사실을 약하게 취급할 수 있다..
![]()
https://arxiv.org/abs/2504.13837 Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?Reinforcement Learning with Verifiable Rewards (RLVR) has recently demonstrated notable success in enhancing the reasoning performance of large language models (LLMs), particularly on mathematics and programming tasks. Similar to how traditional RL helps aarxiv.org Abs..
![]()
https://arxiv.org/abs/2407.21787 Large Language Monkeys: Scaling Inference Compute with Repeated SamplingScaling the amount of compute used to train language models has dramatically improved their capabilities. However, when it comes to inference, we often limit models to making only one attempt at a problem. Here, we explore inference compute as another axisarxiv.org Abstract. Language Model 을 ..
![]()
https://arxiv.org/abs/2605.12474 Reward Hacking in Rubric-Based Reinforcement LearningReinforcement learning with verifiable rewards has enabled strong post-training gains in domains such as math and coding, though many open-ended settings rely on rubric-based rewards. We study reward hacking in rubric-based RL, where a policy is optimizedarxiv.org Abstract. RLVR(Reinforcement Learning with Veri..
![]()
https://arxiv.org/abs/2409.12822 Language Models Learn to Mislead Humans via RLHFLanguage models (LMs) can produce errors that are hard to detect for humans, especially when the task is complex. RLHF, the most popular post-training method, may exacerbate this problem: to achieve higher rewards, LMs might get better at convincing humansarxiv.org Abstract. 언어 모델(LMs)은 인간이 감지하기 어려운 오류를 생성할 수 있으며, 특..
![]()
https://arxiv.org/abs/2210.10760 Scaling Laws for Reward Model OveroptimizationIn reinforcement learning from human feedback, it is common to optimize against a reward model trained to predict human preferences. Because the reward model is an imperfect proxy, optimizing its value too much can hinder ground truth performance, in accorarxiv.org Abstract. 사람의 피드백을 기반으로 하는 RLHF 에서는, 사람의 선호도를 예측하도록 훈..
![]()
https://arxiv.org/abs/2409.11923 Agglomerative Token ClusteringWe present Agglomerative Token Clustering (ATC), a novel token merging method that consistently outperforms previous token merging and pruning methods across image classification, image synthesis, and object detection & segmentation tasks. ATC merges clustarxiv.org Abstract. image classification, image synthesis 그리고 object detection ..
