BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices

Autor: Reuel, Anka, Hardy, Amelia, Smith, Chandler, Lamparth, Max, Hardy, Malcolm, Kochenderfer, Mykel J.

AI models are increasingly prevalent in high-stakes environments, necessitating thorough assessment of their capabilities and risks. Benchmarks are popular for measuring these attributes and for comparing model performance, tracking progress, and ide

Externí odkaz: http://arxiv.org/abs/2411.12990

ReXTrust: A Model for Fine-Grained Hallucination Detection in AI-Generated Radiology Reports

Autor: Hardy, Romain, Kim, Sung Eun, Rajpurkar, Pranav

The increasing adoption of AI-generated radiology reports necessitates robust methods for detecting hallucinations--false or unfounded statements that could impact patient care. We present ReXTrust, a novel framework for fine-grained hallucination de

Externí odkaz: http://arxiv.org/abs/2412.15264

Observation of Cosmic-Ray Anisotropy in the Southern Hemisphere with Twelve Years of Data Collected by the IceCube Neutrino Observatory

Autor: Abbasi, R., Ackermann, M., Adams, J., Agarwalla, S. K., Aguado, T., Aguilar, J. A., Ahlers, M., Alameddine, J. M., Amin, N. M., Andeen, K., Argüelles, C., Ashida, Y., Athanasiadou, S., Axani, S. N., Babu, R., Bai, X., V., A. Balagopal, Baricevic, M., Barwick, S. W., Bash, S., Basu, V., Bay, R., Beatty, J. J., Tjus, J. Becker, Beise, J., Bellenghi, C., BenZvi, S., Berley, D., Bernardini, E., Besson, D. Z., Blaufuss, E., Bloom, L., Blot, S., Bontempo, F., Motzkin, J. Y. Book, Meneguolo, C. Boscolo, Böser, S., Botner, O., Böttcher, J., Braun, J., Brinson, B., Brisson-Tsavoussis, Z., Brostean-Kaiser, J., Brusa, L., Burley, R. T., Butterfield, D., Campana, M. A., Caracas, I., Carloni, K., Carpio, J., Chattopadhyay, S., Chau, N., Chen, Z., Chirkin, D., Choi, S., Clark, B. A., Cochling, C., Coleman, A., Coleman, P., Collin, G. H., Connolly, A., Conrad, J. M., Corley, R., Cowen, D. F., De Clercq, C., DeLaunay, J. J., Delgado, D., Deng, S., Desai, A., Desiati, P., de Vries, K. D., de Wasseige, G., DeYoung, T., Diaz, A., Díaz-Vélez, J. C., Dierichs, P., Dittmer, M., Domi, A., Draper, L., Dujmovic, H., Durnford, D., Dutta, K., DuVernois, M. A., Ehrhardt, T., Eidenschink, L., Eimer, A., Eller, P., Ellinger, E., Mentawi, S. El, Elsässer, D., Engel, R., Erpenbeck, H., Esmail, W., Evans, J., Evenson, P. A., Fan, K. L., Fang, K., Farrag, K., Fazely, A. R., Fedynitch, A., Feigl, N., Fiedlschuster, S., Finley, C., Fischer, L., Fox, D., Franckowiak, A., Fukami, S., Fürst, P., Gallagher, J., Ganster, E., Garcia, A., Garcia, M., Garg, G., Genton, E., Gerhardt, L., Ghadimi, A., Girard-Carillo, C., Glaser, C., Glüsenkamp, T., Gonzalez, J. G., Goswami, S., Granados, A., Grant, D., Gray, S. J., Griffin, S., Griswold, S., Groth, K. M., Guevel, D., Günther, C., Gutjahr, P., Gruchot, K., Ha, C., Haack, C., Hallgren, A., Halve, L., Halzen, F., Hamacher, L., Hamdaoui, H., Minh, M. Ha, Handt, M., Hanson, K., Hardin, J., Harnisch, A. A., Hatch, P., Haungs, A., Häußler, J., Hardy, A., Hayes, W., Helbing, K., Hellrung, J., Hermannsgabner, J., Heuermann, L., Heyer, N., Hickford, S., Hidvegi, A., Hill, C., Hill, G. C., Hmaid, R., Hoffman, K. D., Hori, S., Hoshina, K., Hostert, M., Hou, W., Huber, T., Hultqvist, K., Hünnefeld, M., Hussain, R., Hymon, K., Ishihara, A., Iwakiri, W., Jacquart, M., Jain, S., Janik, O., Jansson, M., Jeong, M., Jin, M., Jones, B. J. P., Kamp, N., Kang, D., Kang, W., Kang, X., Kappes, A., Kappesser, D., Kardum, L., Karg, T., Karl, M., Karle, A., Katil, A., Katz, U., Kauer, M., Kelley, J. L., Khanal, M., Zathul, A. Khatee, Kheirandish, A., Kiryluk, J., Klein, S. R., Kobayashi, Y., Kochocki, A., Koirala, R., Kolanoski, H., Kontrimas, T., Köpke, L., Kopper, C., Koskinen, D. J., Koundal, P., Kowalski, M., Kozynets, T., Krieger, N., Krishnamoorthi, J., Kruiswijk, K., Krupczak, E., Kumar, A., Kun, E., Kurahashi, N., Lad, N., Gualda, C. Lagunas, Lamoureux, M., Larson, M. J., Lauber, F., Lazar, J. P., Lee, J. W., DeHolton, K. Leonard, Leszczyńska, A., Liao, J., Lincetto, M., Liu, Y. T., Liubarska, M., Love, C., Lu, L., Lucarelli, F., Luszczak, W., Lyu, Y., Madsen, J., Magnus, E., Mahn, K. B. M., Makino, Y., Manao, E., Mancina, S., Mand, A., Sainte, W. Marie, Mariş, I. C., Marka, S., Marka, Z., Marsee, M., Martinez-Soler, I., Maruyama, R., Mayhew, F., McNally, F., Mead, J. V., Meagher, K., Mechbal, S., Medina, A., Meier, M., Merckx, Y., Merten, L., Mitchell, J., Montaruli, T., Moore, R. W., Morii, Y., Morse, R., Moulai, M., Moy, A., Mukherjee, T., Naab, R., Nakos, M., Naumann, U., Necker, J., Negi, A., Neste, L., Neumann, M., Niederhausen, H., Nisa, M. U., Noda, K., Noell, A., Novikov, A., Pollmann, A. Obertacke, O'Dell, V., Olivas, A., Orsoe, R., Osborn, J., O'Sullivan, E., Palusova, V., Pandya, H., Park, N., Parker, G. K., Parrish, V., Paudel, E. N., Paul, L., Heros, C. Pérez de los, Pernice, T., Peterson, J., Pizzuto, A., Plum, M., Pontén, A., Popovych, Y., Rodriguez, M. Prado, Pries, B., Procter-Murphy, R., Przybylski, G. T., Pyras, L., Raab, C., Rack-Helleis, J., Rad, N., Ravn, M., Rawlins, K., Rechav, Z., Rehman, A., Resconi, E., Reusch, S., Rhode, W., Riedel, B., Rifaie, A., Roberts, E. J., Robertson, S., Rodan, S., Roellinghoff, G., Rongen, M., Rosted, A., Rott, C., Ruhe, T., Ruohan, L., Ryckbosch, D., Safa, I., Saffer, J., Salazar-Gallegos, D., Sampathkumar, P., Sandrock, A., Santander, M., Sarkar, S., Savelberg, J., Savina, P., Schaile, P., Schaufel, M., Schieler, H., Schindler, S., Schlickmann, L., Schlüter, B., Schlüter, F., Schmeisser, N., Schmidt, E., Schmidt, T., Schneider, J., Schröder, F. G., Schumacher, L., Schwirn, S., Sclafani, S., Seckel, D., Seen, L., Seikh, M., Seo, M., Seunarine, S., Myhr, P. Sevle, Shah, R., Shefali, S., Shimizu, N., Silva, M., Simmons, A., Skrzypek, B., Smithers, B., Snihur, R., Soedingrekso, J., Søgaard, A., Soldin, D., Soldin, P., Sommani, G., Spannfellner, C., Spiczak, G. M., Spiering, C., Stachurska, J., Stamatikos, M., Stanev, T., Stezelberger, T., Stürwald, T., Stuttard, T., Sullivan, G. W., Taboada, I., Ter-Antonyan, S., Terliuk, A., Thiesmeyer, M., Thompson, W. G., Thorpe, A., Thwaites, J., Tilav, S., Tollefson, K., Tönnis, C., Toscano, S., Tosi, D., Trettin, A., Turcotte, R., Elorrieta, M. A. Unland, Upadhyay, A. K., Upshaw, K., Vaidyanathan, A., Valtonen-Mattila, N., Vandenbroucke, J., van Eijndhoven, N., Vannerom, D., van Santen, J., Vara, J., Varsi, F., Veitch-Michaelis, J., Venugopal, M., Vereecken, M., Carrasco, S. Vergara, Verpoest, S., Veske, D., Vijai, A., Walck, C., Wang, A., Weaver, C., Weigel, P., Weindl, A., Weldert, J., Wen, A. Y., Wendt, C., Werthebach, J., Weyrauch, M., Whitehorn, N., Wiebusch, C. H., Williams, D. R., Witthaus, L., Wolf, M., Woodward, H., Wrede, G., Xu, X. W., Yanez, J. P., Yildizci, E., Yoshida, S., Young, R., Yu, S., Yuan, T., Zegarelli, A., Zhang, S., Zhang, Z., Zhelnin, P., Zilberman, P., Zimmerman, M.

We analyzed the 7.92$\times 10^{11}$ cosmic-ray-induced muon events collected by the IceCube Neutrino Observatory from May 13, 2011, when the fully constructed experiment started to take data, to May 12, 2023. This dataset provides an up-to-date cosm

Externí odkaz: http://arxiv.org/abs/2412.05046

'All that Glitters': Approaches to Evaluations with Unreliable Model and Human Annotations

Autor: Hardy, Michael

"Gold" and "ground truth" human-mediated labels have error. The effects of this error can escape commonly reported metrics of label quality or obscure questions of accuracy, bias, fairness, and usefulness during model evaluation. This study demonstra

Externí odkaz: http://arxiv.org/abs/2411.15634

Supernova bounds on new scalars from resonant and soft emission

Autor: Hardy, Edward, Sokolov, Anton, Stubbs, Henry

We study supernova cooling constraints on new light scalars that mix with the Higgs, couple only to nucleons, or couple only to leptons. We show that in all these cases scalars with masses smaller than the plasma frequency in the supernova core are e

Externí odkaz: http://arxiv.org/abs/2410.17347