Innocent explains at some length in Sermo IV. «IN CONSECRATIONE PONTIFICIS», that the matrimony can only exist between legitimate persons (solus consensus inter legitimas personas efficit matrimonium), and therefore, by the sin of infidelity, the spiritual matrimonial union between the Roman Pontiff and the Church would cease to exist; and thus the heretic, no longer a legitimate spouse, would cease to be pope, and could therefore be judged by men. Thus he explains:
For reason of fornication the Roman Church could dismiss the Roman Pontiff. I do not mean carnal fornication but spiritual; and since it is not carnal but a spiritual union, that is because of the error of infidelity; because « he who does not believe has already been judged (John iii) : » and in this article is understood what is read in the Gospel, which you have heard: «You are the salt of the earth, but if the salt should lose its savour wherewith shall it be salted? (Matth. V.) »[92]
The reason for this, he explains in Sermo II, “Faith is so necessary for me, that while I have God alone for a judge; but for committing the sin against faith, I can be judged by the Church, because he who does not believe has already been judged (John3: 18)”[93]
https://www.thecatholicmonitor.com/2023/01/pope-innocent-iii-spiritual-matrimonial.html