Modeling Response Processes to Achieve an Appropriate Factor Structure for the Bafacalo Logical Reasoning Test
DOI:
https://doi.org/10.14738/assrj.1310.12492Abstract
Responses to reasoning tests may reflect not only the ability the tests were designed to measure but also response processes distinct from that ability, which can obscure their factor structure when left unmodeled. The BAFACALO Logical Reasoning Test has been openly available since 2021, but the previous factorial study relied on specific factors derived from an exploratory analysis and on an estimator that treats dichotomous items as continuous, and its model fell just short of acceptable fit. As a syllogism-based test, it is open to response processes such as illicit conversion, the atmosphere effect, and acquiescence. This study reexamines its factorial validity by modeling these response processes, with data from 292 Brazilian public high school students. Five confirmatory factor models were compared: a unidimensional model, the bifactor model of the previous study, and three bifactor models with theory-driven specific factors for illicit conversion and the atmosphere effect, for acquiescence, and for both. Models were estimated with WLSMV and, for comparison, ML and MLMV. No model reached acceptable fit with ML or MLMV. With WLSMV, only the models including illicit conversion and the atmosphere effect were acceptable, and the model that also included acquiescence was selected. The general factor (logical reasoning) and the illicit conversion and atmosphere effect factor showed acceptable reliability, whereas the acquiescence factor did not. Modeling these response processes yielded an appropriate factor structure for the test, separating them from logical reasoning. These findings strengthen the evidence on the factor structure of this openly available test.
References
Alves, F. A., Flores, R. P., Gomes, C. M. A., & Golino, H. F. (2012). Preditores do rendimento escolar: Inteligência geral e crenças sobre ensino-aprendizagem. Revista E-Psi, 1, 97–117. https://www.researchgate.net/publication/235642286_Preditores_do_Rendimento_Escolar_Inteligencia_Geral_e_Crencas_sobre_Ensino-Aprendizagem
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.
Bagozzi, R. P., & Yi, Y. (1988). On the evaluation of structural equation models. Journal of the Academy of Marketing Science, 16(1), 74–94. https://doi.org/10.1007/BF02723327
Begg, I., & Denny, J. P. (1969). Empirical reconciliation of atmosphere and conversion interpretations of syllogistic reasoning errors. Journal of Experimental Psychology, 81(2), 351–354. https://doi.org/10.1037/h0027770
Bentler, P. M. (2009). Alpha, dimension-free, and model-based internal consistency reliability. Psychometrika, 74(1), 137–143. https://doi.org/10.1007/s11336-008-9100-1
Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). The Guilford Press.
Calvin, C. M., Deary, I. J., Fenton, C., Roberts, B. A., Der, G., Leckenby, N., & Batty, G. D. (2011). Intelligence in youth and all-cause-mortality: Systematic review with meta-analysis. International Journal of Epidemiology, 40(3), 626–644. https://doi.org/10.1093/ije/dyq190
Carroll, J. B. (1945). The effect of difficulty and chance success on correlations between items or between tests. Psychometrika, 10(1), 1–19. https://doi.org/10.1007/BF02289789
Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. Cambridge University Press. https://doi.org/10.1017/CBO9780511571312
Chapman, L. J., & Chapman, J. P. (1959). Atmosphere effect re-examined. Journal of Experimental Psychology, 58(3), 220–226. https://doi.org/10.1037/h0041961
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504. https://doi.org/10.1080/10705510701301834
Deary, I. J., Strand, S., Smith, P., & Fernandes, C. (2007). Intelligence and educational achievement. Intelligence, 35(1), 13–21. https://doi.org/10.1016/j.intell.2006.02.001
Golino, H. F., & Gomes, C. M. A. (2014). Psychology data from the “BAFACALO project: The Brazilian Intelligence Battery based on two state-of-the-art models – Carroll’s Model and the CHC model”. Journal of Open Psychology Data, 2(1), e6. https://doi.org/10.5334/jopd.af
Gomes, C. M. A. (2010a). Avaliando a avaliação escolar: Notas escolares e inteligência fluida. Psicologia em Estudo, 15(4), 841–849. https://doi.org/10.1590/S1413-73722010000400020
Gomes, C. M. A. (2010b). Estrutura fatorial da Bateria de Fatores Cognitivos de Alta-Ordem (BaFaCalo). Avaliação Psicológica, 9(3), 449–459. http://pepsic.bvsalud.org/scielo.php?pid=S1677-04712010000300011&script=sci_arttext
Gomes, C. M. A. (2011). Validade do conjunto de testes da habilidade de memória de curto-prazo (CTMC). Estudos de Psicologia (Natal), 16(3), 235–242. https://doi.org/10.1590/S1413-294X2011000300005
Gomes, C. M. A. (2012). Validade de construto do conjunto de testes de inteligência cristalizada (CTIC) da bateria de fatores cognitivos de alta-ordem (BaFaCAlO). Gerais: Revista Interinstitucional de Psicologia, 5(2), 294–316. http://pepsic.bvsalud.org/scielo.php?pid=S1983-82202012000200009&script=sci_arttext
Gomes, C. M. A., & Araujo, J. (2025a). Ciência aberta e democratização de testes: Disponibilização gratuita e aberta do Teste de Velocidade Perceptiva 2 da Bateria de Fatores Cognitivos de Alta Ordem (BAFACALO). Revista Contemporânea, 5(2), e7431. https://doi.org/10.56083/RCV5N2-038
Gomes, C. M. A., & Araujo, J. (2025b). Democratizando os testes de inteligência: Acesso aberto e gratuito ao Teste de Dobraduras da Bateria de Fatores Cognitivos de Alta-Ordem (BAFACALO). European Journal of Education Studies, 12(1), 109–136. https://doi.org/10.46827/ejes.v12i1.5771
Gomes, C. M. A., & Araujo, J. (2025c). Democratizando os testes de inteligência: Acesso aberto e gratuito ao Teste de Indução da Bateria de Fatores Cognitivos de Alta-Ordem (BAFACALO). Observatorio de la Economía Latinoamericana, 23(1), 1–28. https://doi.org/10.55905/oelv23n1-136
Gomes, C. M. A., & Araujo, J. (2025d). Teste de Velocidade Perceptiva 3 da Bateria de Fatores Cognitivos de Alta Ordem (BAFACALO): Disponibilização gratuita e aberta. Cuadernos de Educación y Desarrollo, 17(3), e7727. https://doi.org/10.55905/cuadv17n3-031
Gomes, C. M. A., & Araujo, J. (2026). Incorporating test design into factorial validity: A bifactor reanalysis of the BAFACALO Folding Test. REMUNOM, 13(21), 1–17. https://doi.org/10.66104/00fx1964
Gomes, C. M. A., & Borges, O. N. (2009a). Propriedades psicométricas do conjunto de testes da habilidade visuoespacial. Psico-USF, 14(1), 19–34. https://doi.org/10.1590/S1413-82712009000100004
Gomes, C. M. A., & Borges, O. (2009b). Qualidades psicométricas do conjunto de testes de inteligência fluida. Avaliação Psicológica, 8(1), 17–32. http://pepsic.bvsalud.org/scielo.php?pid=S1677-04712009000100003&script=sci_arttext
Gomes, C. M. A., de Araújo, J., Ferreira, M. G., & Golino, H. F. (2014). The validity of the Cattell-Horn-Carroll model on the intraindividual approach. Behavioral Development Bulletin, 19(4), 22–30. https://doi.org/10.1037/h0101078
Gomes, C. M. A., Farias, H. B., & Jelihovschi, E. G. (2022). Approaches to learning does matter to predict academic achievement. Revista de Psicología, 40(2), 905–933. https://doi.org/10.18800/psico.202202.010
Gomes, C. M. A., & Golino, H. F. (2012a). O que a inteligência prediz: Diferenças individuais ou diferenças no desenvolvimento acadêmico? Psicologia: Teoria e Prática, 14(1), 126–139. http://pepsic.bvsalud.org/pdf/ptp/v14n1/v14n1a10.pdf
Gomes, C. M. A., & Golino, H. F. (2012b). Validade incremental da Escala de Abordagens de Aprendizagem (EABAP). Psicologia: Reflexão e Crítica, 25(4), 400–410. https://doi.org/10.1590/S0102-79722012000400001
Gomes, C. M. A., & Golino, H. (2015). Factor retention in the intra-individual approach: Proposition of a triangulation strategy. Avaliação Psicológica, 14(2), 273–279. https://doi.org/10.15689/ap.2015.1402.12
Gomes, C. M. A., Marques, E. L. L., & Golino, H. F. (2014). Validade incremental dos estilos legislativo, executivo e judiciário em relação ao rendimento escolar. Revista E-Psi, 2, 31–46. https://revistaepsi.com/artigo/2013-2014-ano3-volume2-artigo3/
Gomes, C. M. A., & Nascimento, D. F. (2021a). A medida da habilidade de fluência do modelo CHC: Apresentando o Teste de Fluência Ideativa 2 da Bateria de Fatores Cognitivos de Alta-Ordem (BAFACALO) [Preprint]. ResearchGate. https://doi.org/10.13140/RG.2.2.35726.28481/1
Gomes, C. M. A., & Nascimento, D. F. (2021b). Acesso aberto ao Teste de Fluência Figural da Bateria de Fatores Cognitivos de Alta-Ordem (BAFACALO) como medida da habilidade ampla de fluência do modelo CHC de inteligência [Preprint]. ResearchGate. https://doi.org/10.13140/RG.2.2.15593.62564/1
Gomes, C. M. A., & Nascimento, D. F. (2021c). Acesso aberto e gratuito ao Conjunto de Testes de Inteligência Fluida: Teste de Raciocínio Geral da Bateria de Fatores Cognitivos de Alta-Ordem (BAFACALO) [Preprint]. ResearchGate. https://doi.org/10.13140/RG.2.2.30509.61921/1
Gomes, C. M. A., & Nascimento, D. F. (2021d). Acesso aberto e gratuito ao Teste de Fluência Ideativa 1 da BAFACALO [Preprint]. ResearchGate. https://doi.org/10.13140/RG.2.2.24821.09442/3
Gomes, C. M. A., & Nascimento, D. F. (2021e). Apresentando o Teste de Flexibilidade de Fechamento da BAFACALO [Preprint]. ResearchGate. https://doi.org/10.13140/RG.2.2.31920.28164
Gomes, C. M. A., & Nascimento, D. F. (2021f). Disponibilizando de forma gratuita e aberta o Teste de Memória Associativa 1 da Bateria de Fatores Cognitivos de Alta-Ordem (BAFACALO) [Preprint]. ResearchGate. https://doi.org/10.13140/RG.2.2.29964.03201/1
Gomes, C. M. A., & Nascimento, D. F. (2021g). Disponibilizando de forma gratuita e aberta o Teste de Velocidade Numérica da BAFACALO [Preprint]. ResearchGate. https://doi.org/10.13140/RG.2.2.24114.94407/1
Gomes, C. M. A., & Nascimento, D. F. (2021h). Medidas de inteligência cristalizada: Disponibilizando o Teste de Compreensão Verbal 2 da Bateria de Fatores Cognitivos de Alta-Ordem (BAFACALO) [Preprint]. ResearchGate. https://doi.org/10.13140/RG.2.2.36085.09447/1
Gomes, C. M. A., & Nascimento, D. F. (2021i). Medindo a habilidade de rapidez cognitiva do modelo CHC: Apresentando o Teste de Velocidade Perceptiva 1 da BAFACALO [Preprint]. ResearchGate. https://doi.org/10.13140/RG.2.2.28564.83848/1
Gomes, C. M. A., & Nascimento, D. F. (2021j). Projeto de acesso aberto e gratuito à Bateria de Fatores Cognitivos de Alta-Ordem (BAFACALO): O Teste de Compreensão Verbal 1 do Conjunto de Testes de Inteligência Cristalizada [Preprint]. ResearchGate. https://doi.org/10.13140/RG.2.2.22663.32165/1
Gomes, C. M. A., & Nascimento, D. F. (2021k). Projeto de acesso aberto e gratuito aos testes do LAICO: Teste de Raciocínio Lógico da Bateria de Fatores Cognitivos de Alta-Ordem (BAFACALO) [Preprint]. ResearchGate. https://doi.org/10.13140/RG.2.2.25476.45445/1
Gomes, C. M. A., & Nascimento, D. F. (2021l). Projeto de acesso aos testes de inteligência da BAFACALO: Teste de Compreensão Verbal 3 [Preprint]. ResearchGate. https://doi.org/10.13140/RG.2.2.10499.84001/2
Gomes, C. M. A., & Nascimento, D. F. (2021m). Projeto de acesso da BAFACALO: Teste de Memória Associativa 2 [Preprint]. ResearchGate. https://doi.org/10.13140/RG.2.2.23253.14565/1
Gomes, C. M. A., & Nascimento, D. F. (2021n). Teste de Memória Visual da Bateria de Fatores Cognitivos de Alta-Ordem (BAFACALO) [Preprint]. ResearchGate. https://doi.org/10.13140/RG.2.2.33319.47529
Gomes, C. M. A., Nascimento, D. F., & Araujo, J. (2021a). Acesso aberto ao Teste de Dobraduras (VZ) da BAFACALO [Preprint]. ResearchGate. https://doi.org/10.13140/RG.2.2.21853.95201/2
Gomes, C. M. A., Nascimento, D. F., & Araujo, J. (2021b). Medindo a inteligência fluida: O Teste de Indução da Bateria de Fatores Cognitivos de Alta-Ordem (BAFACALO) [Preprint]. ResearchGate. https://doi.org/10.13140/RG.2.2.17087.84641/3
Gomes, C. M. A., Nascimento, D. F., & Araujo, J. (2021c). Projeto de testes gratuitos e abertos do LAICO: Teste de Velocidade Perceptiva 3 da BAFACALO [Preprint]. ResearchGate. https://doi.org/10.13140/RG.2.2.36278.42563/2
Gomes, C. M. A., Nascimento, D. F., & Araujo, J. (2021d). Teste de Velocidade Perceptiva 2 da Bateria de Fatores Cognitivos de Alta-Ordem (BAFACALO): Disponibilização aberta e gratuita aos testes de medida de rapidez cognitiva do LAICO [Preprint]. ResearchGate. https://doi.org/10.13140/RG.2.2.29567.53928/1
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., & Rosseel, Y. (2021). semTools: Useful tools for structural equation modeling (Version 0.5-5) [R package]. https://CRAN.R-project.org/package=semTools
Li, C.-H. (2016). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods, 48(3), 936–949. https://doi.org/10.3758/s13428-015-0619-7
Marsh, H. W., Hau, K.-T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler's (1999) findings. Structural Equation Modeling, 11(3), 320–341. https://doi.org/10.1207/s15328007sem1103_2
McDonald, R. P. (1999). Test theory: A unified treatment. Lawrence Erlbaum Associates.
McGrew, K. S. (2009). CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research. Intelligence, 37(1), 1–10. https://doi.org/10.1016/j.intell.2008.08.004
Meade, A. W., Johnson, E. C., & Braddy, P. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93(3), 568–592. https://doi.org/10.1037/0021-9010.93.3.568
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D. P., Hesse, B., Humphreys, M., … Yarkoni, T. (2015). Promoting an open research culture. Science, 348(6242), 1422–1425. https://doi.org/10.1126/science.aab2374
Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of personality and social psychological attitudes (pp. 17–59). Academic Press. https://doi.org/10.1016/B978-0-12-590241-0.50006-X
R Core Team. (2024). R: A language and environment for statistical computing (Version 4.4.1) [Computer software]. R Foundation for Statistical Computing. https://www.R-project.org/
Raykov, T. (2001). Estimation of congeneric scale reliability using covariance structure analysis with nonlinear constraints. British Journal of Mathematical and Statistical Psychology, 54(2), 315–323. https://doi.org/10.1348/000711001159582
Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02
Savalei, V. (2018). On the computation of the RMSEA and CFI from the mean-and-variance corrected test statistic with nonnormal data in SEM. Multivariate Behavioral Research, 53(3), 419–429. https://doi.org/10.1080/00273171.2018.1455142
Savalei, V. (2021). Improving fit indices in structural equation modeling with categorical data. Multivariate Behavioral Research, 56(3), 390–407. https://doi.org/10.1080/00273171.2020.1717922
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262–274. https://doi.org/10.1037/0033-2909.124.2.262
Schneider, W. J., & McGrew, K. S. (2018). The Cattell–Horn–Carroll theory of cognitive abilities. In D. P. Flanagan & E. M. McDonough (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (4th ed., pp. 73–163). The Guilford Press.
Spearman, C. (1904). "General intelligence," objectively determined and measured. The American Journal of Psychology, 15(2), 201–292. https://doi.org/10.2307/1412107
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Cristiano Mauro Assis Gomes, Jhonys de Araujo
This work is licensed under a Creative Commons Attribution 4.0 International License.
