Thetis Development Build 2.10.3.6 dev_4 available

User avatar
ramdor
Posts: 1512
Joined: Wed Jul 03, 2019 3:07 pm
Contact:

Thetis Development Build 2.10.3.6 dev_4 available

Postby ramdor » Fri Jul 05, 2024 3:57 am

Hi folks,

Warren has added some additional CW options for the ANAN-G2, and also some new interesting NR2 features. See notes below. The release is here : https://github.com/ramdor/Thetis/releas ... .3.6-dev_4

There are also some additional options for kenwood AI (broadcast) via cat1-4 and tcp/ip cat. I missed them from the change log, so just a heads up.

Cheers.

v2.10.3.6 Change Log - WIP dev_4

[add] CW Edge Length: This is currently only for the ANAN-G2. The ability to load CW edges into the firmware for the legacy ANAN products is not available at this time. The minimum Edge Length is 15ms which, based upon testing, is required to meet the new ARRL Guidelines. However, due to the edge shape, a 15ms total edge yields about 5ms in the range of 10% - 90% amplitude, i.e., the 15ms number is a bit misleading.

[add] New NR2 features: (1) the "Trained" gain method as described below, (2) an alternative Noise Power Estimation method called "NSTAT" which should have a faster response to changing noise levels, and (3) code added in the AE Filter that removes an annoying rumble that could occasionally occur.

["Trained"] The new "Trained" NR has a file (zetaHat.bin). It is a data file that contains the essential information extracted from "training" the algorithm. The algorithm is currently "trained" with about 72 hours of noise and speech recordings. The file may, at some point, be replaced or augmented if different training is required.
Attachments
nr2_trained.png
nr2_trained.png (44.2 KiB) Viewed 2008 times
Last edited by ramdor on Fri Jul 05, 2024 5:03 pm, edited 3 times in total.
Trucker
Posts: 349
Joined: Wed Nov 03, 2021 5:16 pm

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby Trucker » Fri Jul 05, 2024 1:09 pm

Is this new noise reduction selectable? Or does it replace NR2?
I have used a couple of AI based noise reduction applications and while they worked quite well, they also added latency to the receive audio. ( this was with my Flex 6600M)
I will download and give it a test run.
James
WD5GWY
Trucker
Posts: 349
Joined: Wed Nov 03, 2021 5:16 pm

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby Trucker » Fri Jul 05, 2024 3:01 pm

I changed the settings in this screen and it seemed the audio was a bit clearer. But, I had to raise the AGC level because I could still hear background hiss in the audio.
I will keep trying different settings just in case there is something I have missed.
James
WD5GWY
20240705_091722.jpg
20240705_091722.jpg (4.96 MiB) Viewed 2069 times
w3ub
Posts: 109
Joined: Fri Dec 14, 2018 6:32 pm
Location: Lunenburg, NS/Florida

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby w3ub » Fri Jul 05, 2024 3:24 pm

I have found that by using the Trained Threshold (at the default -0.5), any noise is reduced even when setting the AGC threshold down well below the noise. Using Gamma at that AGC threshold always resulted in some noise.

Currently running the trained threshold and NSTAT method, seems to be OK.

Doug
w3ub
Posts: 109
Joined: Fri Dec 14, 2018 6:32 pm
Location: Lunenburg, NS/Florida

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby w3ub » Fri Jul 05, 2024 3:25 pm

I meant to add, that in this case it is almost like the VSQL was active (it is not).
Doug
Trucker
Posts: 349
Joined: Wed Nov 03, 2021 5:16 pm

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby Trucker » Fri Jul 05, 2024 3:42 pm

I don't mean to keep replying to myself, but, I selected "Trained" and the background hiss went completely away! I am listening to some locals on 40 meters and they are just like full quieting FM.
There are some thunderstorms a couple of hundred miles from me. I can see the noise floor jumping. Yet, I cannot hear the static crashes breaking through like normal. I can tune off frequency and turn the volume way up and still no background noise.
This is what I experienced with RM Noise that I tested on my Flex 6600M. But, the latency was very noticeable since RM Noise uses an off site server to process the audio. Plus, it uses fixed receive filters.
The AI Noise Reduction that is now in Thetis is 100% better.
James
WD5GWY
Trucker
Posts: 349
Joined: Wed Nov 03, 2021 5:16 pm

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby Trucker » Fri Jul 05, 2024 3:46 pm

w3ub wrote:I have found that by using the Trained Threshold (at the default -0.5), any noise is reduced even when setting the AGC threshold down well below the noise. Using Gamma at that AGC threshold always resulted in some noise.

Currently running the trained threshold and NSTAT method, seems to be OK.

Doug

Doug,
I didn't see your reply before posting my previous post. But, yes, this is a great improvement to the noise reduction in Thetis. Honestly, I didn't think it could get any better than what we had.
Boy, was I wrong!
I did leave the Threshold setting at the default as well.
James
WD5GWY
User avatar
W2PA
Posts: 170
Joined: Sun Apr 09, 2017 6:34 pm
Location: LaGrangeville, NY
Contact:

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby W2PA » Fri Jul 05, 2024 4:49 pm

If you haven't yet tried the Trained Threshold version of NR2, make time and do so. You'll be amazed. I haven't tinkered with the Threshold setting much yet, but with the default at -0.5 it is - well, I already said it - amazing.

Setup-DSP-NR/ANF tab - click the radio button for Trained Threshold in the NR2 panel.
73,
Chris, W2PA
Trucker
Posts: 349
Joined: Wed Nov 03, 2021 5:16 pm

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby Trucker » Fri Jul 05, 2024 5:10 pm

W2PA wrote:If you haven't yet tried the Trained Threshold version of NR2, make time and do so. You'll be amazed. I haven't tinkered with the Threshold setting much yet, but with the default at -0.5 it is - well, I already said it - amazing.

Setup-DSP-NR/ANF tab - click the radio button for Trained Threshold in the NR2 panel.

I did finally try this after reading Doug's post. It makes a real difference.
Thanks for the reply.
James
WD5GWY

What will be interesting to see is,if this feature makes it into PiHPSDR for the G2 Saturn radios. I don't know how much of an additional processing load this would put on the CM4 module.
A couple of guys I know have the G2 Saturn and say that NR2 in PiHPSDR works well.
I have an old V1 controller that has an older Pi in it. And NR2 in PiHPSDR in it works quite well.
StefanoAlvaro
Posts: 23
Joined: Thu Nov 24, 2022 3:56 pm

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby StefanoAlvaro » Sat Jul 06, 2024 8:21 am

New DNR eally very interesting, a new unique and innovative step in the typologies of DNR!!
It is very comfortable, but unfortunately the low signals are covered, even with a threshold of -5. I hope that can increase its sensitivity. Really congratulations!!
I think that no one uses normal NR anymore (at least I think) and in my opinion it could be inserted in its place, rather than entering the DSP menu
ww1wxradio
Posts: 5
Joined: Tue Jul 02, 2024 1:17 pm

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby ww1wxradio » Sat Jul 06, 2024 11:08 am

I noticed with this version that my VOX quit working. Can someone check theirs to see if it is just me or is it a bug in this version

Steve . . . WW1WX
RobB
Posts: 22
Joined: Thu Jun 02, 2022 3:11 pm

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby RobB » Sat Jul 06, 2024 12:28 pm

ww1wxradio wrote:I noticed with this version that my VOX quit working. Can someone check theirs to see if it is just me or is it a bug in this version

Steve . . . WW1WX

VOX is working for me.
ww1wxradio
Posts: 5
Joined: Tue Jul 02, 2024 1:17 pm

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby ww1wxradio » Sat Jul 06, 2024 1:54 pm

OK Thanks . . . Figured it out . . . If I have VAC1 on to put sound through computer the vox does not work. If I shut off VAC1 it works fine.
w3ub
Posts: 109
Joined: Fri Dec 14, 2018 6:32 pm
Location: Lunenburg, NS/Florida

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby w3ub » Sat Jul 06, 2024 4:28 pm

After playing some more, I have dropped the trained threshold to -3.0, which is adequate to remove just about all noise, but will allow for very low signals to still come through. In addition, the Auto AGC is not set to 3, which is much lower than I could run before. This also helps with low signals, and you can get away with it all the time with the improvement in NR2 performance.

Doug
User avatar
W1AEX
Posts: 431
Joined: Sun Apr 09, 2017 6:17 pm
Location: Connecticut, USA
Contact:

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby W1AEX » Sat Jul 06, 2024 5:01 pm

I will echo comments about the amazing performance of NR2 running in Trained mode. Works beautifully and as previously mentioned it is like listening to FM with a squelch. I would agree with Stefano's comment as I do notice that the default -0.5 setting does cover up some moderately weak signals. I have found that a threshold level set to the lowest setting of -5.0 seems to let them through without passing background noise. I still need to tune around some more to find out how well a weak signal can get through at that setting.

This NR implementation is certainly superior to anything I have ever used so thanks to Warren for another amazing addition to Thetis!

Rob W1AEX
Last edited by W1AEX on Sun Jul 07, 2024 2:07 am, edited 1 time in total.
"One thing I am certain of is that there is too much certainty in the world."
Joe
Posts: 191
Joined: Tue Aug 22, 2017 5:58 pm

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby Joe » Sat Jul 06, 2024 8:47 pm

Just wanted to say Thank You to all that made this happen. You gotta hear this, it is great! I used the adjusted settings mentioned above for a start.

Thanks and 73’s
Joe
WD5Y
NC3Z
Posts: 478
Joined: Sun Oct 29, 2017 8:57 pm
Location: Merritt, NC

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby NC3Z » Sat Jul 06, 2024 10:16 pm

Unless I missed it are there any articles or docs that explain these now algos?
Gary NC3Z
va7qi
Posts: 115
Joined: Mon Apr 10, 2017 4:22 am
Location: Quadra Island, BC
Contact:

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby va7qi » Sat Jul 06, 2024 11:14 pm

One word: AMAZING!. 73 & tnx de va7qi, ....Erik.
JohnFT61
Posts: 77
Joined: Wed May 17, 2023 8:13 pm

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby JohnFT61 » Sun Jul 07, 2024 2:52 am

I have in the past been using AUTO AGC. With this version (and no RF input), it displays 77 for the AUTO AGC value. Much lower than when using a previous version. With the previous version (2.10.3.5-u2), with AUTO AGC selected, it displayed 101. Nobody has mentioned this yet, is everyone using manual AGC?


73
N2KBE
User avatar
ramdor
Posts: 1512
Joined: Wed Jul 03, 2019 3:07 pm
Contact:

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby ramdor » Sun Jul 07, 2024 9:20 am

@JohnFT61 there was a change in v2.10.3.6_dev_2, which is no doubt the reason. Info in the change logs.
W1SWL
Posts: 16
Joined: Wed Feb 08, 2023 6:46 pm

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby W1SWL » Sun Jul 07, 2024 9:57 am

Richie:
I was just wondering what is next? Can we train the algorithm ourselves with our own local conditions and operating environment(s) ie Antenna Systems?

Or perhaps even "add" to the existing training file?

I guess I would like to understand what "training" actually is.
Thanks, Art W1SWL
User avatar
rbduck
Posts: 329
Joined: Tue Dec 03, 2019 1:49 pm

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby rbduck » Sun Jul 07, 2024 11:06 am

As I started reading this thread I thought "what! how could NR2 possibly be approved upon?" I downloaded and installed this latest development and the answer was clear. Really clear. NR2 has been surpassed all my expectations. Now where do we go? I'm sure that the developers have ideas in the back of their minds and it will be brought to the forefront in the days to come. Thank you to Richie and all the others for making these the truly the best transceivers on the market!! They are a pleasure to operate. Thanks again!
73
Ruben
NB4R
Apache-Labs Anan 7000DLE MKII Black -- Thetis 2.10.3.6 dev_3 -- Windows 11
Warren_NR0V
Posts: 14
Joined: Sun Apr 09, 2017 9:59 pm

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby Warren_NR0V » Sun Jul 07, 2024 1:55 pm

I’ve had multiple requests for more information on the operation of the new “Trained” NR2 mode. I’ll give a very brief summary here.
This mode, like other NR2 modes, operates in the frequency domain, i.e., after performing an FFT on the incoming data. In each frame of data, each frequency bin is evaluated and included or excluded in the output based upon its signal-to-noise ratio. The purpose of the training is to “teach” the algorithm to triangulate on the actual signal-to-noise ratios based upon multiple estimates of them that can be calculated in real-time. The training is the critical item here and the most difficult to get right. I MAY attempt some trainings with different input data, and perhaps even operator-selectable training choices, in the future. However, no commitments at this point. I hope you enjoy the new mode and I’ll be reading the feedback for future ideas.
73,
Warren NR0V
StefanoAlvaro
Posts: 23
Joined: Thu Nov 24, 2022 3:56 pm

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby StefanoAlvaro » Mon Jul 08, 2024 8:38 am

I think this is a step that marks a new generation of technology in the ham world, leaving light years behind everything that until yesterday could have come close to thetis. I am truly honored to have been able to witness the birth of this technology!


My feedback is that, the limit is in very low (at the same level as the noise floor and 1-3 dB higher) but still understandable voices/signal, which are cut or completely covered. By setting the threshold to -5 (but already starting from -3.9) the normal background noise reappears, but the voice remains fragmented or covered. By removing NR or returning to “gamma” the voice becomes intelligible again. Unfortunately I don't know what it could depend on because I don't know how the algorithm works but if I can be useful in any way I'll make everything I can do available.
I also have a question, what does “operates in the frequency domain” mean? If there were some link (not too difficult) to learn more about, I would gladly read it.
Warren_NR0V
Posts: 14
Joined: Sun Apr 09, 2017 9:59 pm

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby Warren_NR0V » Mon Jul 08, 2024 12:14 pm

Hi Stefano & All,

I'm looking at a possible adjustment that might allow better access to signals when the signal-to-noise ratio is very low. If that works out, you'll hear more about it later.

With regard to "frequency domain" processing, if you search for "frequency domain" or "time domain versus frequency domain" you should find lots of articles and some videos. When we sample a signal with an ADC, we get a sequence of samples taken at different times (time domain). We can then perform a Discrete Fourier Transform (DFT), or Fast Fourier Transform (FFT), on a sequence of those samples and they are changed to a set of "bins" that represent different frequencies (frequency domain) rather than representing different times. There are many algorithms that can better and more efficiently do processing in this frequency-domain than in the time-domain, for example a filter where we just include some frequencies and exclude others. When finished with our frequency-domain processing, an Inverse-DFT or Inverse-FFT will convert the frequency bins back to time samples.

73,
Warren NR0V
User avatar
KA1GXR
Posts: 48
Joined: Sun Apr 09, 2017 10:53 pm
Location: ASHLAND MA

Re: Thetis Development Build 2.10.3.6 dev_4 available

Postby KA1GXR » Mon Jul 08, 2024 3:37 pm

I found the same results with lower strength signals as Stefano and tried same settings to improve. I use AM a lot and New NR2 seemed to have more difficulty keeping audio levels up . Possibly samples used for developement were mostly ssb and other narrower modes than AM so understandable but it did help. Very nice to use it as a squelch to monitor a frequency until activity occurs in any mode. Fun testing it.
Tom
KA1GXR

Return to “Thetis”