Thetis Development Build 2.10.3.6 dev_4 available

[ramdor]

Hi folks,

Warren has added some additional CW options for the ANAN-G2, and also some new interesting NR2 features. See notes below. The release is here : https://github.com/ramdor/Thetis/releas ... .3.6-dev_4

There are also some additional options for kenwood AI (broadcast) via cat1-4 and tcp/ip cat. I missed them from the change log, so just a heads up.

Cheers.

v2.10.3.6 Change Log - WIP dev_4

[add] CW Edge Length: This is currently only for the ANAN-G2. The ability to load CW edges into the firmware for the legacy ANAN products is not available at this time. The minimum Edge Length is 15ms which, based upon testing, is required to meet the new ARRL Guidelines. However, due to the edge shape, a 15ms total edge yields about 5ms in the range of 10% - 90% amplitude, i.e., the 15ms number is a bit misleading.

[add] New NR2 features: (1) the "Trained" gain method as described below, (2) an alternative Noise Power Estimation method called "NSTAT" which should have a faster response to changing noise levels, and (3) code added in the AE Filter that removes an annoying rumble that could occasionally occur.

["Trained"] The new "Trained" NR has a file (zetaHat.bin). It is a data file that contains the essential information extracted from "training" the algorithm. The algorithm is currently "trained" with about 72 hours of noise and speech recordings. The file may, at some point, be replaced or augmented if different training is required.

[Trucker]

Is this new noise reduction selectable? Or does it replace NR2?
I have used a couple of AI based noise reduction applications and while they worked quite well, they also added latency to the receive audio. ( this was with my Flex 6600M)
I will download and give it a test run.
James
WD5GWY

[Trucker]

I changed the settings in this screen and it seemed the audio was a bit clearer. But, I had to raise the AGC level because I could still hear background hiss in the audio.
I will keep trying different settings just in case there is something I have missed.
James
WD5GWY

20240705_091722.jpg (4.96 MiB) Viewed 5919 times

[w3ub]

I have found that by using the Trained Threshold (at the default -0.5), any noise is reduced even when setting the AGC threshold down well below the noise. Using Gamma at that AGC threshold always resulted in some noise.

Currently running the trained threshold and NSTAT method, seems to be OK.

Doug

[w3ub]

I meant to add, that in this case it is almost like the VSQL was active (it is not).
Doug

[Trucker]

I don't mean to keep replying to myself, but, I selected "Trained" and the background hiss went completely away! I am listening to some locals on 40 meters and they are just like full quieting FM.
There are some thunderstorms a couple of hundred miles from me. I can see the noise floor jumping. Yet, I cannot hear the static crashes breaking through like normal. I can tune off frequency and turn the volume way up and still no background noise.
This is what I experienced with RM Noise that I tested on my Flex 6600M. But, the latency was very noticeable since RM Noise uses an off site server to process the audio. Plus, it uses fixed receive filters.
The AI Noise Reduction that is now in Thetis is 100% better.
James
WD5GWY

[Trucker]

I have found that by using the Trained Threshold (at the default -0.5), any noise is reduced even when setting the AGC threshold down well below the noise. Using Gamma at that AGC threshold always resulted in some noise.

Currently running the trained threshold and NSTAT method, seems to be OK.

Doug

Doug,
I didn't see your reply before posting my previous post. But, yes, this is a great improvement to the noise reduction in Thetis. Honestly, I didn't think it could get any better than what we had.
Boy, was I wrong!
I did leave the Threshold setting at the default as well.
James
WD5GWY

[W2PA]

If you haven't yet tried the Trained Threshold version of NR2, make time and do so. You'll be amazed. I haven't tinkered with the Threshold setting much yet, but with the default at -0.5 it is - well, I already said it - amazing.

Setup-DSP-NR/ANF tab - click the radio button for Trained Threshold in the NR2 panel.

[Trucker]

If you haven't yet tried the Trained Threshold version of NR2, make time and do so. You'll be amazed. I haven't tinkered with the Threshold setting much yet, but with the default at -0.5 it is - well, I already said it - amazing.

Setup-DSP-NR/ANF tab - click the radio button for Trained Threshold in the NR2 panel.

I did finally try this after reading Doug's post. It makes a real difference.
Thanks for the reply.
James
WD5GWY

What will be interesting to see is,if this feature makes it into PiHPSDR for the G2 Saturn radios. I don't know how much of an additional processing load this would put on the CM4 module.
A couple of guys I know have the G2 Saturn and say that NR2 in PiHPSDR works well.
I have an old V1 controller that has an older Pi in it. And NR2 in PiHPSDR in it works quite well.

[StefanoAlvaro]

New DNR eally very interesting, a new unique and innovative step in the typologies of DNR!!
It is very comfortable, but unfortunately the low signals are covered, even with a threshold of -5. I hope that can increase its sensitivity. Really congratulations!!
I think that no one uses normal NR anymore (at least I think) and in my opinion it could be inserted in its place, rather than entering the DSP menu

[ww1wxradio]

I noticed with this version that my VOX quit working. Can someone check theirs to see if it is just me or is it a bug in this version

Steve . . . WW1WX

[RobB]

I noticed with this version that my VOX quit working. Can someone check theirs to see if it is just me or is it a bug in this version

Steve . . . WW1WX

VOX is working for me.

[ww1wxradio]

OK Thanks . . . Figured it out . . . If I have VAC1 on to put sound through computer the vox does not work. If I shut off VAC1 it works fine.

[w3ub]

After playing some more, I have dropped the trained threshold to -3.0, which is adequate to remove just about all noise, but will allow for very low signals to still come through. In addition, the Auto AGC is not set to 3, which is much lower than I could run before. This also helps with low signals, and you can get away with it all the time with the improvement in NR2 performance.

Doug

[W1AEX]

I will echo comments about the amazing performance of NR2 running in Trained mode. Works beautifully and as previously mentioned it is like listening to FM with a squelch. I would agree with Stefano's comment as I do notice that the default -0.5 setting does cover up some moderately weak signals. I have found that a threshold level set to the lowest setting of -5.0 seems to let them through without passing background noise. I still need to tune around some more to find out how well a weak signal can get through at that setting.

This NR implementation is certainly superior to anything I have ever used so thanks to Warren for another amazing addition to Thetis!

Rob W1AEX

[Joe]

Just wanted to say Thank You to all that made this happen. You gotta hear this, it is great! I used the adjusted settings mentioned above for a start.

Thanks and 73’s
Joe
WD5Y

[NC3Z]

Unless I missed it are there any articles or docs that explain these now algos?

[va7qi]

One word: AMAZING!. 73 & tnx de va7qi, ....Erik.

[JohnFT61]

I have in the past been using AUTO AGC. With this version (and no RF input), it displays 77 for the AUTO AGC value. Much lower than when using a previous version. With the previous version (2.10.3.5-u2), with AUTO AGC selected, it displayed 101. Nobody has mentioned this yet, is everyone using manual AGC?


73
N2KBE

[ramdor]

@JohnFT61 there was a change in v2.10.3.6_dev_2, which is no doubt the reason. Info in the change logs.

[W1SWL]

Richie:
I was just wondering what is next? Can we train the algorithm ourselves with our own local conditions and operating environment(s) ie Antenna Systems?

Or perhaps even "add" to the existing training file?

I guess I would like to understand what "training" actually is.
Thanks, Art W1SWL

[rbduck]

As I started reading this thread I thought "what! how could NR2 possibly be approved upon?" I downloaded and installed this latest development and the answer was clear. Really clear. NR2 has been surpassed all my expectations. Now where do we go? I'm sure that the developers have ideas in the back of their minds and it will be brought to the forefront in the days to come. Thank you to Richie and all the others for making these the truly the best transceivers on the market!! They are a pleasure to operate. Thanks again!

[Warren_NR0V]

I’ve had multiple requests for more information on the operation of the new “Trained” NR2 mode. I’ll give a very brief summary here.
This mode, like other NR2 modes, operates in the frequency domain, i.e., after performing an FFT on the incoming data. In each frame of data, each frequency bin is evaluated and included or excluded in the output based upon its signal-to-noise ratio. The purpose of the training is to “teach” the algorithm to triangulate on the actual signal-to-noise ratios based upon multiple estimates of them that can be calculated in real-time. The training is the critical item here and the most difficult to get right. I MAY attempt some trainings with different input data, and perhaps even operator-selectable training choices, in the future. However, no commitments at this point. I hope you enjoy the new mode and I’ll be reading the feedback for future ideas.
73,
Warren NR0V

[StefanoAlvaro]

I think this is a step that marks a new generation of technology in the ham world, leaving light years behind everything that until yesterday could have come close to thetis. I am truly honored to have been able to witness the birth of this technology!


My feedback is that, the limit is in very low (at the same level as the noise floor and 1-3 dB higher) but still understandable voices/signal, which are cut or completely covered. By setting the threshold to -5 (but already starting from -3.9) the normal background noise reappears, but the voice remains fragmented or covered. By removing NR or returning to “gamma” the voice becomes intelligible again. Unfortunately I don't know what it could depend on because I don't know how the algorithm works but if I can be useful in any way I'll make everything I can do available.
I also have a question, what does “operates in the frequency domain” mean? If there were some link (not too difficult) to learn more about, I would gladly read it.

[Warren_NR0V]

Hi Stefano & All,

I'm looking at a possible adjustment that might allow better access to signals when the signal-to-noise ratio is very low. If that works out, you'll hear more about it later.

With regard to "frequency domain" processing, if you search for "frequency domain" or "time domain versus frequency domain" you should find lots of articles and some videos. When we sample a signal with an ADC, we get a sequence of samples taken at different times (time domain). We can then perform a Discrete Fourier Transform (DFT), or Fast Fourier Transform (FFT), on a sequence of those samples and they are changed to a set of "bins" that represent different frequencies (frequency domain) rather than representing different times. There are many algorithms that can better and more efficiently do processing in this frequency-domain than in the time-domain, for example a filter where we just include some frequencies and exclude others. When finished with our frequency-domain processing, an Inverse-DFT or Inverse-FFT will convert the frequency bins back to time samples.

73,
Warren NR0V

[KA1GXR]

I found the same results with lower strength signals as Stefano and tried same settings to improve. I use AM a lot and New NR2 seemed to have more difficulty keeping audio levels up . Possibly samples used for developement were mostly ssb and other narrower modes than AM so understandable but it did help. Very nice to use it as a squelch to monitor a frequency until activity occurs in any mode. Fun testing it.
Tom
KA1GXR