I found the answer, so I'm posting it for future reference.
Windows uses Active Probing to detect whether internet connectivity is OK (details).
It basically probes some servers using DNS, HTTP, UDP, IPv6, etc. queries to detect the connectivity mode.
WindowsSpyBlocker is a tool which checks the issues with this. I ran it, and selected option 1 (Telemetry) and then option 2 (NCSI = Network Connectivity Status Indicator):

Choosing option 5 (Test the internet connection) shows the error:

It seems that there are IPv6 and UDP problems between my network and Microsoft's default server (msftconnecttest.com).
So, I picked option 3 (Apply Firefox NCSI), and then checked again with option 5. It now works like a charm:

The network indicator is now OK, and Windows Update / Microsoft Store work as expected.
The relevant Registry Key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NlaSvc\Parameters\Internet, after applying FireFox NCSI, looks like this:

textual version (registry update file) of the picture above:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NlaSvc\Parameters\Internet]
"ActiveDnsProbeContent"="208.67.222.222"
"ActiveDnsProbeContentV6"="2620:119:35::35"
"ActiveDnsProbeHost"="resolver1.opendns.com"
"ActiveDnsProbeHostV6"="resolver1.opendns.com"
"ActiveWebProbeContent"="success"
"ActiveWebProbeContentV6"="success"
"ActiveWebProbeHost"="detectportal.firefox.com"
"ActiveWebProbeHostV6"="detectportal.firefox.com"
"ActiveWebProbePath"="success.txt"
"ActiveWebProbePathV6"="success.txt"
"CaptivePortalTimer"=dword:00000000
"CaptivePortalTimerBackOffIncrementsInSeconds"=dword:00000005
"CaptivePortalTimerMaxInSeconds"=dword:0000001e
"EnableActiveProbing"=dword:00000001
"PassivePollPeriod"=dword:0000000f
"StaleThreshold"=dword:0000001e
"WebTimeout"=dword:00000023
Save into a file registry_update.reg, double-click and run the update.