آخرین تغییرات IBSng

از ویکی پارس پویش
(تفاوت بین نسخه‌ها)
پرش به: ناوبری, جستجو
سطر ۱۷: سطر ۱۷:
 
| C_staging || 372 || ۱۳۹۶/۰۳/۲۷
 
| C_staging || 372 || ۱۳۹۶/۰۳/۲۷
 
|-
 
|-
−
| مثال || مثال || مثال
+
| C_shahkar || 370 || ۹۵/۱۱/۲۴
 
|-
 
|-
−
| مثال || مثال || مثال
+
| C_mongo || 374 || ۱۳۹۶/۰۳/۲۷
 
|-
 
|-
−
| مثال || مثال || مثال
+
| C_diameter || 402 || ۱۳۹۶/۰۷/۲۶
 
|-
 
|-
−
| مثال || مثال || مثال
+
| C_pmacct || 373 || ۱۳۹۶/۰۳/۲۷
 
|-
 
|-
−
| مثال || مثال || مثال
+
| C_diameter_mongo || 402 || ۱۳۹۶/۰۷/۲۶
 
|-
 
|-
−
| مثال || مثال || مثال
+
| C_lan_mongo || 372 || ۱۳۹۶/۰۳/۲۷
 
|-
 
|-
−
| مثال || مثال || مثال
+
| C_lan_acc_staging || 372 || ۱۳۹۶/۰۳/۲۷
 
|-
 
|-
−
| مثال || مثال || مثال
+
| C_lan_diameter || 402 || ۱۳۹۶/۰۷/۲۶
 
|-
 
|-
−
| مثال || مثال || مثال
+
| C_lan_diameter_mongo || 402 || ۱۳۹۶/۰۷/۲۶
−
|-
+
−
| مثال || مثال || مثال
+
 
|}
 
|}
 
=''' C Branch '''=
 
=''' C Branch '''=

نسخهٔ ‏۱۲ نوامبر ۲۰۱۷، ساعت ۱۳:۱۴


محتویات


Branch Name Tag No Release Date
C_lan_acc 376 ۱۳۹۶/۰۷/۰۳
C 376 ۱۳۹۶/۰۷/۰۳
C_staging 372 ۱۳۹۶/۰۳/۲۷
C_shahkar 370 ۹۵/۱۱/۲۴
C_mongo 374 ۱۳۹۶/۰۳/۲۷
C_diameter 402 ۱۳۹۶/۰۷/۲۶
C_pmacct 373 ۱۳۹۶/۰۳/۲۷
C_diameter_mongo 402 ۱۳۹۶/۰۷/۲۶
C_lan_mongo 372 ۱۳۹۶/۰۳/۲۷
C_lan_acc_staging 372 ۱۳۹۶/۰۳/۲۷
C_lan_diameter 402 ۱۳۹۶/۰۷/۲۶
C_lan_diameter_mongo 402 ۱۳۹۶/۰۷/۲۶

C Branch

تگ 374 تا 376 در تاریخ ۱۳۹۶/۰۷/۰۳

Support Request: #1555 Add User Import Log

Improvement: #1617 Add PubSub to Invoice Profile

Feature: #1553 Add PubSub to Custom Field 

This commit is copy/paste of PR No. 23 which has been done by Morteza
Nourelahi Alamdari on development branch. Because this is a priority
for us in stable branch we create this commit.

Improvement: #1618 Correct Space between Codes

Feature: #1239 Add `online_payment.getAvailableGatewayTypes` Method 

This method have been copy/paste from staging branch because it's been
needed for Wispot project and we can't wait for merging staging branch
to stable one.

Feature: #1611 Add Forgot Password to User Web Panel 

Actually this feature has been implemented before in staging branch by
Saeed Rasooli. Since we are not ready now to merge our staging branch
to stable one, so we have some how copy/paste the changes (with some
cleanup) from staging branch to answer customer request.

Improvement: #1618 Return admin_id in addNewAdmin Method

Bugfix: #1515 fix all online graph's time line in user and admin panel 

all the time line issues in both admin panel and user panel for real time graphs have been fixed for branches without mongoDB.


تگ 372 تا 374 در تاریخ ۱۳۹۶/۰۵/۲۳

Feature: #1146 Add New Permission "CHANGE USER LIMIT MAC" 

 From now on, only admins with specified permission can modify the
 limit mac attribute of users. This permission has "CHANGE USER
 ATTRIBUTES" permission as it's parent.

Feature: #1462 add failed online reason to user information page 

 if there was a reason for failed online it will be shown on user information page .

Bugfix: #1417 Fix Load Balancing Functionality of IPPool Plugin 

 The `ippool_main.getLoader().getIPpoolByID` will return either an
 "IPPool" or "LoadBalancingIPPool" object. In later case the object has
 no `getContainer` method. Instead we use `setIPInPacket` method which
 is available in both objects and has the same functionality.

Feature: #1317 Add ISP Filter to `getListOfCharges` Charge Action

Feature: #1317 Add ISP Filter to `getListOfGroups` Group Action

Bugfix :#1436 change QOS defs propertis to accept string input

Bugfix :#1424 real time bw graph timeline problem 

 time line problem solved using utc time guide.

Improvement: don't log wrong password

Bugfix: #1367 search_expired_user.php check_all_users problem 

 multi select problem for users has been handled.

Bugfix: #1408 Port call_id_blacklist plugin to C branch

Fix: Show Exact Value for ISP's on Online Users

Feature : #1361 change the sub service qos name to limited drop down 

 change  the sub service qos name to limited drop down options .

Feature : #1360 edit charge priority from 1-10 to 1-50 

 edit charge priority to 1-50 and it's backend error and validation rules.


تگ 371 تا 372 در تاریخ ۱۳۹۶/۰۳/۲۳

- Bugfix: #1339 Load Invoice Main in Online Worker

- Bugfix : #1300 delete realtime request log interface 

 real time request log  deleted  from interface because the source code of this link  had been deleted before .

- Bugfix: #1339 Load Invoice Main in Online Worker

- Bugfix: #1278 Fix Account Not Started Counter 

 When "account not started" counter got negative value,
 IBSng web panel shown "0*" value that means negative number.
 The main reason for this issue was when a user had several sub services,
 it tried to decrease counter per each of them (when their requests received),
 while it increased counter once on authentication action. So to solve this issue,
 we handled __decreamentNotStartedCounter method which check loaded user's
 instance that must be Master to run decrease action.
 Example diagram for this issue:
  Step1: Authentication +1   << Correct
  Step2: Master         -1   << Correct
  Step3: Free-Site      -1   << Incorrect
  StepN: SubServiceN    -1   << Incorrect
  StepX: SubServiceX    -1   << Incorrect

- Bugfix: #1292 Fix Bulk Kill Action in Case of Choosing More Than One Group 

 When user chooses more than one group/isp/etc in Search User panel and
 selects bulk kill action it raises and error. It was because of
 invalid format of passing parameters.

- Bugfix: Fix Minor Typo Error

- Bugfix: #1235 Fix Minor Invoicing DB Bug 

 Column reference to 'pi_issue_date' was ambiguous!

- Bugfix: #1228 Fix Max Time Usage Charge From Custom Field 

 Resolving 'max_rule_time_usage' charg rule to be read from user
 custome field (e.g: 'F:time') haven't been handled.

- Bugfix: #1215 Fix Timezone Issue in Online Graph


تگ 360 تا 371 در تاریخ ۱۳۹۵/۱۲/۱۰

- Bugfix: Fix Active Leaves 

 The `getActiveLeaves` method must be called in 'MAIN' process.

- BugFix: Various Internal Relogin fixes

- Improvement: Make sure listen loop won't exit on an abnormal exception

- BugFix: Setting max rule traffic usage value as a float in custom fields would have break max rule traffic plugin

- Improvement: Disable Partitioning Check Contraints 

 This is done because of some efficiency issues

- BugFix: We should ignore negative threshold when QV is passed to cisco isg

- BugFix: A mid session reauthorize after a restart will create an instance without start accounting

- BugFix: Do not panic if charge rule is None during stop session

- BugFix: Not saving user after logout will lead to inconsistent user object

- BugFix: Take credit limit into account when calculating remaining bytes. This fixes postpaid user problem with isg prepaid services

- Improvement: Kill max delay default value was too large. Set it to 2 hours by default

- Improvement: Make auth rate control optional and turn it off by default

- Bugfix: #1449 Fix `users.group_id` Bug 

Fixed `getDBColName` in a way which returns the part after the dot,
e.g 'usersgroup_id' => 'group_id'

- Bugfix: #1449 Fix Ambiguous Fields `user_id` and `group_id` in Management Summary Report 

 After adding two columns `user_id` and `group_id` to `connection_log`
 table it created a conflict with `users` table on in management
 summary queries; in this fix we change SQL queries to exactly specify
 which fields we mean by `user_id` and `group_id`: `users.user_id`,
 `users.group_id`, etc.

- Bugfix: add missing function call on commit of periodic acc plugin

- Bugfix: #1503 Fix MAC Extraction From RAS of ZTE Type 

 ZTE model rases change the format of user mac address in recent
 framework update, e.g:
 old format: `78d99fd34e4b`
 new format: `78:d9:9f:d3:4e:4b`

- Improvement: Few logging for radius process

- BugFix: Reauthorize bytes were deducted twice from remaining bytes

- BugFix: Do not start accounting on reauthorize. Removes lots of nagging about online dic not found in error log

- Improvement: Make sure we never return a quota of 0 in case of a successful reauthorize

- Improvement: Run internal relogin with same priority as auth. Long queues might cause WS request to return while task has not been done yet

- BugFix: reauthorize_used_bytes should be used bytes from last update and not from start of session. Fixed premature QV0 while user still has credit.

- Bugfix: don't assign ip to sub service

- Bugfix: skip users without ignore_ras

- Feature: Add support for cisco_isg_max_reauthorize_mbytes and cisco_isg_max_reauthorize_minutes attributes to force reauthorize after a certain threshold

- Feature: Multiple prepaid services per account support

- BugFix: Max Rule Traffic Usage were not working correctly with multiple rules sharing one value (and custom field) and multiple instances using them

- BugFix: Onlines table in single user info page were became messed up after first ajax based refresh

- BugFix: Multi login did not work with Multi Service rases other than ISG

- Bugfix: #1452 Fix/Update Deprecated Links in Graph Menu 

 The following links are deprecated and removed:
 All Realtime Graph
 Internet Realtime Graph
 Voip Realtime Graph
 BW Realtime Graph
 Also updated modified links

- Improvement: Failure to change sub services (ex. failed COA) should be detected. Also resync services after internal relogin to make sure we have correct services

- Bugfix: #1136 Fix Name of Import File in Import User Menu 

 Note: after changing the log process to pass the logging mechanism to
 python syslog module, user_import logs won't be written to a local
 file which will cause to "View Import Log" link on "Import User" page
 after uploading CSV file won't work. this can be fixed by adding new
 log type (aside `QueueLogger` and `DirectLogger`) to IBS logger
 module, but this is not a priority now; look at the comments on
 changed files for more info.

- Bugfix: cast ignore ras ids to int

- Bugfix: read min/max event time directly from defs

- Bugfix: fix group usages of connection analysis

- Improvement: Remove `price` Filter From User ID Attribute

- Feature: Add User ID Attribute to Search User Page

- Feature: optionally do autoRenew on abs expired user

- Bugfix: log deposit_type in deposit change

- Cleanup: clean md5 response

تگ 360 در تاریخ ۱۳۹۵/۱۰/۲۹

  • Bug Fixes​

1. Re-Onlined users might have get failed status due to left over from previous sessions

2. Radius Attributes with integer values were not working in charge rule attribute radius

3. Removed Extra print

4. Incorrect call to getUserAttrs caused exception

5. Some initial variables where missing in configuration file and advanced configuration

6. Undefined session raised unnecessary exception

7. Threshold Bytes/Seconds did not trigger internal reauthorize due to bad merge

8. Second username change fail in case of internal relogin

  • Improvement

1. Allow Acct-Interim-Interval to be passed to failed users

2. Exception handling for internal relogin

3. API cleanups for cisco isg rases

4. get method for SNMP client

تگ 359 در تاریخ ۱۳۹۵/۱۰/۲۹

Bugfix: fix internal relogin answer to main proc

Bugfix: Race condition might cause internal relogin to fail

Bugfix: Autorenew would fail if user has next group and current group does not have group_credit

Bugfix: fix limit port binding

Bugfix: fix periodic acc failed login

Bugfix: fix non-working periodic accounting plugin

Bugfix: Some mikrotik devices send mschapv2 flags as non-zero which is not compliant by standard but leads to wrong passwords

Improvement: add zero credit limit to search user

Bugfix: Make sure we never get blocked infinitely in relogin process

Bugfix: Do not fetch in/out bytes in case accounting not started to prevent lots of nagging about online dic not found with ISG prepaid services

Cleanup: Internal Relogin Cleanups

Bugfix: Try to retry query in case of deadlock

Feature: Add PubSub to Extra Charge and MC Modules

Improvement: Allow relogin to be called in blocking mode, so caller will not return until relogin is finished

Bugfix: Race condition caused internal relogin to raise error

Feature: add a plugin to ignore checking limit port based on incoming ras

Bugfix: Internal Relogin were not working correctly with multiple concurrent requests

Bugfix: Threshold might not be defined in session yet

Bugfix: ChangeCredit Set and AutoRenew was not working with new Internal Relogin

Bugfix: Internal Relogin support for changeCredit, updateUserAttrs and setOneChargeRuleUsage to prevent incorrect calculations during online sessions

Bugfix: Clear internal reauthorize threshold, if it is not required anymore

Bugfix: Renew should be called while user instances has been temporarily logged off

Bugfix: Length of DummyPacket were not working

Bugfix: Critical bug in new sub services which caused partial logs not to work properly

Bugfix: allow God admin to send SMS

Bugfix: ignore updating None user object

Feature: Ability to fetch ip-to-mac mapping from a redis database

Bugfix: default value for field references

Bugfix: set online status base on online layout data

Bugfix: up/down cpmb doesn't support field value

Bugfix: hide negative acct not started (business care, we should fix root cause in core)

Bugfix: fix getting charge rule value from user

Bugfix: fix night-free extra charge

Bugfix: fix active leave list

Bugfix: fix setting attribute field in max_rule_traffic_usage

Bugfix: Various new ISG service fixes

Improvement: Log traceback if an online session dictionary has been found

Bugfix: wait for reload after creditChange

Bugfix: ISG Minor Fixes

Bugfix: fix non-working script to recreate bw tree

Feature: Add charging name and qos name to service definition of charge rules

Bugfix: Prevent error if we receive a slave stop accounting without having a session on master isg ras

Cleanup: Cleaned up redis backend section of configuration file

Improvement: Better error handling of expired ras storage dict items

Bugfix: ACS2 ras unique ids were changing for every call. Changed it to use authenticator of incomding packet instead

Bugfix: dhcp attributes should be available in slave aps as it might have individual dhcp ippools

Bugfix: Delete and ttl method of ras storage were not working

Bugfix: Cisco ISG fixes to work with new redis based ras storage

Bugfix: Delete and ttl method of ras storage were not working

Bugfix: fix bulk user deposit

Feature: show charge rules of each service in separate tab

Bugfix: fix non-working home graphs again, issue #1190

Bugfix: DHCP IP assignment was not working correctly if multiple online workers were used

Bugfix: fix graphs in interface (replace realtime stuff), issue #1169

Bugfix: add unique id to ras_msg

Bugfix: Peridic Notification: Credit rule: behave like user plugin Check for AUTO RENEW and AUTO RECHARGE eligibilty before sending notification

Bugfix: Cisco ISG bandwidth attributes were reverse

Improvement: Optimized event for better response time

Improvement: Cisco ISG IP EAP Send account logon on receive of a session start event

Improvement: When syncing subscriber services on cisco ISG only deactivate/activate services which has been changed.

Improvement: Better calculation of ISG bandwidth bursts

Bugfix: Session Dict for Primay service were not created on start accounting

Bugfix: Multiple fixes in cisco ISG EAP ras

Improvement: Use redis ttl instead of iterate and cleanup

Bugfix: Fixed Typo in log console

Bugfix: Minor fixes and cleanups

Bugfix: Change Service with empty service list should not be sent

Improvement: Log activated services in log console while authenticating

Bugfix: Fixed conflicting attributes between mvts and cisco

Bugfix: If user was re-online after rule change occured, we would have never switch services. Current approch will sync services upon re-online using cisco isg session query

Feature: Switched ras storage from lmdb to redis

Bugfix: EAP ras state dictionary was growing without bound

Bugfix: user disconnecting from slave ap should trigger logout on ISG ras

Bugfix: Service kill should deactive service

Bugfix: Reonline of new enhanced services were not working correctly

Bugfix: COA log console were not written

Bugfix: log console were not written for change service

Improvement: Do Deactivate and activate coa requests in a single packet

Improvement: Enhanced services tested

Feature: Initial support for enhanced services

Bugfix: Internal Reauthorize were not working in ISG rases

Bugfix: Can stay runner were not working properly

Bugfix: internet reauthorize was not using can stay runner

Bugfix: Reauthorize packet on cisco 10k series does no pass any cisco-avpair

Bugfix: Service kill did not initiate master session kill

Bugfix: Fixed half written partials caused duplicated deletion processs which might not be necessary

Bugfix: Fixed Typo

Improvement: Various minor enhancements

Improvement: Killing a service in POD style should KILL parent session

Bugfix: Minor cisco ISG fixes

Feature: Add cisco_isg_ios_software cisco attribute with options IOS and IOSXR

Bugfix: Fixed typo which might cause prepaid reauthorize to fail before credit finish

Bugfix: Fixed reauthorize after a restart might cause incorrect values in ras storage

Improvement: Renamed remaining secs to next event secs to clarify the value

Bugfix: Cisco isg prevent avalanche for tarriff change reauthorizes

Improvement: Improved timing to prevent run/shm from filling up

Improvement: Prevent XSS

Bugfix: Fixed reauthorize caused session to be overwritten

Improvement: Better online session timings to prevent run/shm from filling up

Bugfix: Fixed typo

Improvement: Reauthorize used bytes are passed as total used bytes in this session

Feature: Only god admins are allowed to call runDebugCode

Improvement: Enhancements to radius server

Bugfix: More Cisco ISG Fixes

Improvement: Online user count now separate services from master sessions

Bugfix: Fixed reauthorize credit finish were not working

Improvement: Cisco ISG refactoring to improve code reusability

Bugfix: fix failed login

Feature: Log sub service in log console

Improvement: Sub services should not allocate ip addresses

Improvement: Enhanced remaining seconds calculations

Bugfix: Fixed error message

Bugfix: Fixed ISG change bw were not working

Improvement: Use threshold bytes instead of remaining for cisco isg remaining quota

Improvement: Decreased maximum reauthorize time to 4 hours and 1GB

Bugfix: ISG Fixes

Bugfix: Fixed Cisco ISG ras did not delete some ras storage data leading to full /run/shm

Feature: Added support for method beforeDeleteSessionDict to RasPacketHandler

Bugfix: Fixed Cisco ISG does not allow volumes more than 4GB

Feature: Added saveDecodedData method to be able to manually commit decoded data changes (ex. EAP ras)

Bugfix: Fixed update services is not needed on re-online

Bugfix: Fixed ras msgs without service points to Master session

Bugfix: Cisco ISG fixes for prepaid services

Feature: Allow changing of accounting list in ISG

Improvement: Moved common cisco isg methods into a base class, Cisco isg Portal ras

Bugfix: Cisco ISG Ras Fixes

Bugfix: Cisco ISG PPP Fixes

Feature: Send username during COA

Bugfix: Cisco ISG Ras Fixes

Bugfix: Cisco ISG Slave AP ras to overcome weird behaviour of cisco isg on radius proxy+dhcp configuration

Improvement: Removed testing code form isg ip

Feature: Cisco ISG IP EAP with IBS assigned dhcp addresses and dhcp initiator

Feature: Added real time notification default

Bugfix: Fixed items were returning all lmdb items instead of just target dict

Bugfix: Fixed maclib to support more formats

Improvement: Better debugging

Improvement: Better logging

Improvement: Turned off debug by default

Bugfix: New Plugin API fixes

Bugfix: Fixed unresolved conflict

Bugfix: Fixed syntax error

Bugfix: Fixed conflict between service accounting and prepaid services

Bugfix: Fixed Typo

Bugfix: Fixed a race condition which might cause ISG user service not to accept

Bugfix: Fixed peap inner identifiers were incorrect

Bugfix: EAP Ras Fixes to conform RFC

Bugfix: Fixed tariff change flag were not working

Bugfix: ISG IP EAP Ras Fixes

Cleaup: Removed merge leftovers

Bugfix: Fixed indentation errors caused by merge

Bugfix: Fixed COA attributes were not passed correctly

Bugfix: Fixed primary service passed serveral times

Feature: Reauthorize and tariff change support in charge plugin

Bugfix: Fixed multiple typos

Feature: Added a link pointing on how to create cert files

Bugfix: Fixed multiservice on isg ip eap

Feature: EAPCalc added in addons

Bugfix: Fixed tls keys should be optional

Improvement: Sample config for cisco isg ip eap

Feature: Cisco ISG IP EAP Ras Support

Improvement: EAP Ras Minor Enhancements

Improvement: Minor Enhancements

Improvement: Charge rule cache enhancements

Bugfix: Fixed Reauthorize requests should not create a new session

Bugfix: Fixed Typo

Bugfix: More EAP Fixes

Bugfix: Fixed cisco isg service sessions were initiated in no start accoungint state

Bugfix: Fixed re-online multilogin wasn't correctly work due to subservices

Improvement: Better Logging

Bugfix: Fixed mac and port were not shown correctly

Bugfix: Cisco ISG Fixes

Feature: Cisco ISG PPP Ras in new ras branch

Feature: Added showing sub-service in user info online session layer and connection log

Bugfix: More cisco ISG Fixes

Feature: Added support for cisco ISG services

Improvement: Cisco ISG PPP Enhancements

Feature: Cisco ISG PPP Ras

Feature: In RAS BW for Cisco ISG

C_staging Branch

تگ 371 تا 372 در تاریخ ۱۳۹۶/۰۳/۲۷

- Bugfix: #1339 Load Invoice Main in Online Worker

- Bugfix : #1300 delete realtime request log interface 

 real time request log  deleted  from interface because the source code of this link  had been deleted before .

- Bugfix: #1339 Load Invoice Main in Online Worker

- Bugfix: #1278 Fix Account Not Started Counter 

 When "account not started" counter got negative value,
 IBSng web panel shown "0*" value that means negative number.
 The main reason for this issue was when a user had several sub services,
 it tried to decrease counter per each of them (when their requests received),
 while it increased counter once on authentication action. So to solve this issue,
 we handled __decreamentNotStartedCounter method which check loaded user's
 instance that must be Master to run decrease action.
 Example diagram for this issue:
  Step1: Authentication +1   << Correct
  Step2: Master         -1   << Correct
  Step3: Free-Site      -1   << Incorrect
  StepN: SubServiceN    -1   << Incorrect
  StepX: SubServiceX    -1   << Incorrect

- BugFix: reloadUser without wait for lock caused infinite loop

- BugFix: Online User Storage must be flushed during IBSng start up

- Cleanup: Cleaned up check online method

- BugFix: Fixed reloadUser might incorrectly run while lock has not been acquired

- BugFix: Redis might incorrectly reset pubsub in busy situations

- BugFix: Credit Limit was displayed incorrectly

- BugFix: Mutable object should not be passed as default argument

- BugFix: Fix Relogin User Issue On Update Custom Field

- BugFix: Fix Relogin User Issue On Update Custom Field

- Bugfix: #1292 Fix Bulk Kill Action in Case of Choosing More Than One Group 

 When user chooses more than one group/isp/etc in Search User panel and
 selects bulk kill action it raises and error. It was because of
 invalid format of passing parameters.

- BugFix: mp observer change in isp loader caused isp states to reset

- BugFix: unload/load instead of reload causes race condition where isp object is no longer available

- BugFix: Online user storage were using incorrect redis db

- Bugfix: #1235 Fix Minor Invoicing DB Bug

- Bugfix: #1235 Fix Minor Invoicing DB Bug 

 Column reference to 'pi_issue_date' was ambiguous!

- BugFix: Kairos might be initialized incorrectly because system redis library was imported instead of local one

- Cleanup: Remove Unused Method `__renewMustResetCredit`

- BugFix: Kairos might be initialized incorrectly because system redis library was imported instead of local one

- BugFix: Fixed Typo which cause isp deposit role back not to work correctly

- BugFix: __internetReauthorizeCheckForIncorrectSubService was not working due to typo

- BugFix: Fix memory leak of web service process by replacing cyclic-reference-creator collections.OrderedDict with odict.odict in kairos library

- BugFix: Consume ISP deposit was broken after Internal Relogin. Fix to always reload ISP and log after consuming deposit.

- Bugfix: #1228 Fix Max Time Usage Charge From Custom Field

- Bugfix: #1228 Fix Max Time Usage Charge From Custom Field 

 Resolving 'max_rule_time_usage' charg rule to be read from user
 custome field (e.g: 'F:time') haven't been handled.

- BugFix: Duplicate detector was not caching reply packets and never replay them

- Bugfix: #1215 Fix Timezone Issue in Online Graph

- Improvement: Log next event and kill dict in can stay event of log console

- BugFix: Prevent maximum threshold minutes might have caused avalanche situtation, causing cisco ISG bras to overload

- BugFix: Use Acct-Session-Id in kick user via coa instead of Cisco-Account-Info

- BugFix: Internal relogin cleanup and bug fixes

- Improvement: Increased default max thresholds to decrease the load on ras and ibs

- Cleanup: Change RAS Name 'Wireless' to '4ipnet'

- Cleanup: SOAP Cleanup

- Feature: add send_lock_users capability to notification rules


تگ 369 تا 371 در تاریخ ۱۳۹۵/۱۲/۱۰

- Bugfix: Fix Active Leaves 

 The `getActiveLeaves` method must be called in 'MAIN' process.

- BugFix: We should not add internal_relogin flag to user attributes

- BugFix: Various Internal Relogin fixes

- Improvement: Make sure listen loop won't exit on an abnormal exception

- BugFix: Setting max rule traffic usage value as a float in custom fields would have break max rule traffic plugin

- Bugfix: fix delete notification bug

- Improvement: Disable Partitioning Check Contraints 

 This is done because of some efficiency issues

- Improvement: Disable adding check constraint to partitioned tables

- BugFix: We should ignore negative threshold when QV is passed to cisco isg

- BugFix: A mid session reauthorize after a restart will create an instance without start accounting

- BugFix: Do not panic if charge rule is None during stop session

- BugFix: Not saving user after logout will lead to inconsistent user object

- BugFix: Take credit limit into account when calculating remaining bytes. This fixes postpaid user problem with isg prepaid services

- Improvement: Kill max delay default value was too large. Set it to 2 hours by default

- Improvement: Make auth rate control optional and turn it off by default

- Bugfix: #1449 Fix `users.group_id` Bug 

 Fixed `getDBColName` in a way which returns the part after the dot,
 e.g 'usersgroup_id' => 'group_id'

- Bugfix: #1449 Fix Ambiguous Fields `user_id` and `group_id` in Management Summary Report 

 After adding two columns `user_id` and `group_id` to `connection_log`
 table it created a conflict with `users` table on in management
 summary queries; in this fix we change SQL queries to exactly specify
 which fields we mean by `user_id` and `group_id`: `users.user_id`,
 `users.group_id`, etc.

- Bugfix: #1449 Fix Ambiguous Fields `user_id` and `group_id` in Management Summary Report 

 After adding two columns `user_id` and `group_id` to `connection_log`
 table it created a conflict with `users` table on in management
 summary queries; in this fix we change SQL queries to exactly specify
 which fields we mean by `user_id` and `group_id`: `users.user_id`,
 `users.group_id`, etc.

- Bugfix: add missing function call on commit of periodic acc plugin

- Bugfix: Add `disabled` Column to `online_payment_gateway` Table

- Bugfix: #1503 Fix MAC Extraction From RAS of ZTE Type 

 ZTE model rases change the format of user mac address in recent
 framework update, e.g:
 old format: `78d99fd34e4b`
 new format: `78:d9:9f:d3:4e:4b`

- Improvement: Few logging for radius process

- BugFix: Reauthorize bytes were deducted twice from remaining bytes

- BugFix: Do not start accounting on reauthorize. Removes lots of nagging about online dic not found in error log

- Improvement: Make sure we never return a quota of 0 in case of a successful reauthorize

- Improvement: Run internal relogin with same priority as auth. Long queues might cause WS request to return while task has not been done yet

- BugFix: reauthorize_used_bytes should be used bytes from last update and not from start of session. Fixed premature QV0 while user still has credit.

- BugFix: Fixed bad commit

- Bugfix: #1449 Fix `users.group_id` Bug 

 Fixed `getDBColName` in a way which returns the part after the dot,
 e.g 'usersgroup_id' => 'group_id'

- Bugfix: don't assign ip to sub service

- Bugfix: use ippool interface method fix #1468

- Bugfix: skip users without ignore_ras

- Feature: Add support for cisco_isg_max_reauthorize_mbytes and cisco_isg_max_reauthorize_minutes attributes to force reauthorize after a certain threshold

- Feature: Multiple prepaid services per account support

- BugFix: Max Rule Traffic Usage were not working correctly with multiple rules sharing one value (and custom field) and multiple instances using them

- BugFix: Onlines table in single user info page were became messed up after first ajax based refresh

- BugFix: Multi login did not work with Multi Service rases other than ISG

- Bugfix: #1452 Fix/Update Deprecated Links in Graph Menu 

 The following links are deprecated and removed:
 All Realtime Graph
 Internet Realtime Graph
 Voip Realtime Graph
 BW Realtime Graph  
 Also updated modified links

- Improvement: Failure to change sub services (ex. failed COA) should be detected. Also resync services after internal relogin to make sure we have correct services

- Bugfix: #1136 Fix Name of Import File in Import User Menu 

 Note: after changing the log process to pass the logging mechanism to
 python syslog module, user_import logs won't be written to a local
 file which will cause to "View Import Log" link on "Import User" page
 after uploading CSV file won't work. this can be fixed by adding new
 log type (aside `QueueLogger` and `DirectLogger`) to IBS logger
 module, but this is not a priority now; look at the comments on
 changed files for more info.

- BugFix: Re-Onlined users might have get failed status due to left over from previous sessions

- BugFix: Radius Attributes with integer values were not working in charge rule attribute radius

- Improvement: Allow Acct-Interim-Interval to be passed to failed users

- Improvement: Exception handling for internal relogin

- Bugfix: Incorrect call to getUserAttrs caused exception

- BugFix: Some initial variables where missing in configuration file and advanced configuration

- BugFix: Undefined session raised unnecessary exception

- Improvement: API cleanups for cisco isg rases

- Bugfix: cast ignore ras ids to int

- Bugfix: read min/max event time directly from defs

- Improvement: Add Sample Configurations for Work Queue, Pub/Sub and Topic 

 Also did some modification to broker module to work with different
 configurations

- Bugfix: fix group usages of connection analysis

- Improvement: Add Abstraction Layer Over BrokerFactory for Connection Handler

- Improvement: Add Open Channel Callback Function 

 Some initialization should be done after channel opened in
 non-blocking mode.  
 Also add some add callback function to be able to define callbacks
 after conneciton initiation

- Improvement: Remove `price` Filter From User ID Attribute

- Feature: Add User ID Attribute to Search User Page

- BugFix: Threshold Bytes/Seconds did not trigger internal reauthorize due to bad merge

- Feature: Add RabbitMQ Broker

- BugFix: Second username change fail in case of internal relogin

- Improvement: get method for SNMP client

- Bugfix: fix edit invoice rule

- Bugfix: fix online payment

- Bugfix: fix editInvoiceProfile

- Bugfix: fix add/update extra charge profile

- Improvement: Apply Post Rotate Script With Each Rotate

- Improvement: Apply Standard Format to Shahkar Output Files

- Feature: Add ShahakrID and CodeMelli to Shahkar Log

- Feature: Add ShahkarID and CodeMelli to User Attrs 

 These attributes added to 'Comment' section of user attributes

- Improvment: Change Default CSVStorage Splitter and Make PyTables Logging of Radius Server Optional

- Feature: Add CSVStorage to Log Module 

 Also changed 'Shahkar' module to use CSVStorage instead of
 PostgreSQLStorage

- Bugfix: fix auto_recharge_ignore_deposit flag

- Bugfix: fix typo in checking charge usage

- Feature: add group and before credit to connection_log attrs

- Feature: implement rabbitmq interface

- Bugfix: fix failed login for limit port/mac and password)

- Feature: md5 digest

- Bugfix: fix coordinate in shahkar log

- Feature: wireless ras

- Feature: try to get ap_name and coordinate from instance attrs

- Feature: new plugin (autoRenew requires next group)

- Bugfix: fix handling empty/None ras_ids in log console

- Bugfix: fix and complete write/read group and before_credit in connection logs

- Bugfix: convert console buffer to list

- Feature: show kill reason in connection log of user panael

- Bugfix: fix adding msg to buffer

- Bugfix: fix sending ras_loader

- Feature: Save Radius Log in PyTables Structure

- Feature: MC: add 'armaghan' SMS provider, issue #729

- Cleanup: User Location: remove SOAP handler (Location should be removed completely)

- Bugfix: User Import: fix KeyError and cleaning in user_import_actions: __updateImportingStats

- Bugfix: TG: User Import: wrong arguments order in re.sub

- Bugfix: TG: User Import: NameError

- Cleanup: C_invoice_dev: remove Invoice SOAP handler and all WSDL methods (models are there), migrated to JSON-RPC

- Feature: C_invoice: add config param 'change_service_share_isp_groups', used in getServiceSelection

- Bugfix: adapt pytables with new C branch

- Bugfix: SOAPModel: vales with type 'long' were skipped when converting to dict

- Bugfix: SOAPModel: vales with type 'long' were skipped when converting to dict

- Cleanup: remove extra_charge_soap_handler.py, migrate to JSON-RPC

- Doc: fix docstring in voucher_handler.py

- Doc: fix docstring in extra_charge_soap_handler.py

- Bugfix: C_dev: templates: fix again in KILL and CLAER perms, online_users_js.tpl

- Bugfix: fix commit 04111c, related to KILL USER and CLEAR USER again, #940

- Bugfix: C_invoice: service_price plugin: remove checkInput methods, issue #1121 

 modifying group's service price and extra traffic options by non-GOD admin is not working
 permission checking (CHANGE GROUP and CHANGE USER ATTRIBUTES) has been done before

- Feature: Online Payment: add 'Disable' property to Gateway

- Improvement: add JSON-RPC method getGatewayInfo, use it in TG

- Bugfix: allow using user attr 'A:<attr-name>' in charge rule attributes

- Bugfix: C_dev: call transactionFinished in finally block

- Cleanup: C_dev: user_actions.py: yet more cleaning

- Cleanup: C_dev: user_actions.py: more cleanings

- Cleanup: C_dev: user_actions.py: rechargeUsers

- Feature: add flag auto_renew_ignore_deposit: cherry-picked in C_dev

- Cleanup: C_dev: rewrite reset credit / trash remaining decision logic

- added isp names and group names select as general ibs widgets

- Bugfix: irrelevant perm check in bw.getAllLeafNames

- remove realtime graphs & old php image-based graphs, fix timezone bug #888, #1051

- Feature: add user/group flag: Renew Do Not Reset Charge Rule Usage

- Bugfix: fix NameError in core/user/plugins/enable_failed_login.py: NoSectionError

- Feature: C_dev: Notification: add message_type == user_event 

 insert directly to user_events table
 no relation to Message Center

- Bugfix: fix and update Saman gateway WSDL, remove old files 

 rename ReferencePayment.wsdl.old to ReferencePayment_OLD.wsdl

- Cleanup: C_dev: cleaning online_payment_actions.py

- Bugfix: C_dev: user_actions.py: NameError in addNewUsersQuery

- Bugfix: TG: fix bad GroupNameSelecField and ISPNameSelecField

- added isp names and group names select as general ibs widgets

- Bugfix: C_invoice_dev: Fixed Rounding problem with user deposit

- Cleanup: Clean Init Function of Report Cleaner

- Bugfix: C_dev: call transactionFinished in finally block

- Cleanup: C_dev: user_actions.py: yet more cleaning

- Feature: add flag auto_renew_ignore_deposit: cherry-picked in C_dev

- Bugfix: C_invoice_dev: online_payment_arbitrary_convert_formula

- Bugfix: TG: fix bad GroupNameSelecField and ISPNameSelecField

- added isp names and group names select as general ibs widgets

- Bugfix: fix and update Saman gateway WSDL, remove old files 

 rename ReferencePayment.wsdl.old to ReferencePayment_OLD.wsdl

- Cleanup: C_invoice_dev: online_payment_main.py: remove duplocate mellat_shaparak

- Cleanup: C_invoice_dev: cleaning online_payment_actions.py, and compatibility with C_dev

- Bugfix: fix and update Saman gateway WSDL, remove old files

- Bugfix: fix active leave list

- Bugfix: Online Payment: issue #991, fix error formats don't wrap GeneralException in GeneralException again

- Feature: JSON-RPC: add method isp.listISPsWithIDs, issue #993

- Feature: JSON-RPC: add method group.listGroupsWithIDs, issue #993

- Bugfix: MihanSMS: fix stupid and non-working code of __getBodies written by a newbie, issue #874

- Bugfix: Security: fix Bug #1007, add .htaccess file to /util, /inc, /help, IN_user

- Feature: allow performing online payment without any result action(credit/deposit) 

 added for wispot, issue #994
 must put 'no_action' key in web_attributes in preparePayment request

- Cleanup: paymentVerified: argument 'add_type' was ignore and not passed

- Improvement: Online Payment: return Bank Ref ID in Verify 

 return ref_id in verifyPayment and manualVerifyPayment json-rpc methods
 change msg to ref_id in gateway.verify and gateway.manualVerify return values
 issue #989 for wispot

- Bugfix: fix NameError in pasargad_gateway.py: getExceptionText, issue #990

- Feature: add new JSON-RPC method login.webLogin for new integrated web panel, issue #930

- Bugfix: notification: don't set threshold if notify failed 

 return succeed (True/False) in rule.notify

- Bugfix: C_invoice: typo in invoice_controller.py: startwsith->startswith, #976

- Bugfix: C_invoice: autoRecharge: fix stupid logic of checking user_recharge_deposit call transactionFinished in finally block

- Bugfix: notification: fix non-working msg_type='url', issue #680

- Bugfix: C_dev: CRITICAL: unable to change normal username 

 Bug initiated after a cherry-pick of Farshad's old commit by me on 2016-07-16

- add addons/client/reload/reload_defs.py

- Cleanup: move all params from defs.py to core/defs_lib/defs_defaults.py and core/paths.py 

 Feature: allow changing all params (except for paths) from web, by updating db/defs.sql using defs_defaults.py
 Must run db/defs.sql after each update

- Bugfix: update db/defs.sql

- Feature: C_dev: add IranKish gateway (Tejarat Bank), issue #937, not tested on C_dev

- Bugfix: fix non-working MihanSMS mc provider

- Bugfix: fix smarty function canDo, don't add admin_username to params 

 remove unused argument admin_username
 Fixes Bug #940, related to KILL USER and CLEAR USER

- Feature: use custom field in group

- C_invoice: Credit Notiticatio Rule: take {credit + recharge_deposit} instead of {credit + deposit}

- Feature: C_invoice: add config param 'change_service_share_isp_groups', used in getServiceSelection

- Remove Map page (containing outdated or redundant links) from Admin panel -> Home

- Bugfix: Negative Credit Expiration Date: saved Start='1970-01-01' in DB

- Bugfix: SyntaxError in core/notification/notification_rule.py

- Improvement: turbogears: use JSON-RPC instead of XML-RPC

- Online Payment: preparePayment: rename argument 'attribute_map' to 'attributes'

- Online Payment: add mandatory argument 'callback_url' to preparePayment 

 remove gateway attributes for callback/return/revert url (default 127.0.0.1*IBSng-tg), wrong by design

- Cleanup: soap: remove unused complex types related to online payment

- Cleanup: get rid of online_payment_soap_handler, and the rest of online payment related soap methods

- Cleanup: remove unused online payment soap handlers: manualVerifyPayment, manualApplyPayment, purgePayment

- Cleanup: remove unused soap import from /admin/setting/online_payment/online_payment_controller.py

- Cleanup: remove (old) report of Search Pending Payments, admins must use Online Payment Report

- Cleanup: less usage of soap handlers in Admin -> Settings -> Online Payment 

 add getAllGatewayInfos, manualApplyPayment and purgePayment xml-rpc handlers
 rename manualVerify into manualVerifyPayment
 remove manualSettle which is not used

- Bugfix: TG: restoreAuthDataBySessionID was not working, wrong param

- Improvement: json-rpc and xml-rpc: authenticate: INVALID_AUTH_TYPE: show given value

- Cleanup: remove soap handler verifyPayment

- Cleanup: remove soap handler preparePayment

- Cleanup: remove soap handler getPaymentSuggestedAmount

- Cleanup: remove soap handler getAvailablePaymentGateways

- Cleanup: TG: Unify admin and user online payment controllers

- Cleanup: TG: cleaning user/online_payment_controller.py

- Cleanup: TG: online payment: cleaning and trying to abstract user and admin controllers

- Improvement: TG: online payment: show ZarinPal bank Ref ID

- Improvement: TG: online payment: show Saman bank Ref ID

- Cleanup: C_staging: remove "Transfer Deposit" feature, which was disabled in 2012 (tag 137)

- Improvement: parsianCallback: change 'Operation Canceled' message to 'Incomplete Request, no "au"'

- Cleanup: online payment: fix unused xml-rpc methods, add missing ones 

 replace soap with xml-rpc in turbogears user controllers
 and some cleaning in turbogears user controllers

- Cleanup: online payment: don't use soap model in actions

- Cleanup: online_payment_actions: getAvailableGatewayTypes: remove duplicate gateway types, preserving the order

- Cleanup: online payment: add actions.getAvailableGatewayTypes, use it in soap handler

- Cleanup: TG: session.py: use XML-RPC instead of SOAP 

 restoreAuthDataBySessionID is used in online payment callbacks

- Cleanup: TG: replace SOAPError with Exception in @exception_handler(...)

- Cleanup: replace imports of "soap.soap_util.auth" with "ibsng.lib.auth" in turbogears controllers

- Improvement: soap: Map class: add __iter__ method and newFromDictionary classmethod

- Improvement: soap: add __iter__ method to SOAPModel, so that we can convert model into dict simply by dict(model)

- Feature: Online Payment Report: add 'Amount From' and 'Amount To' filters

- Improvement: add 'before_credit' column to connection_log, not tested

- Improvement: add 'group_id' column to connection_log, not tested

- Cleanup: db/functions.sql: break long list of args

- Bugfix: NameError (db_main) in core/user/plugins/mc.py

- Cleanup: remove 'partitioning_pg_min_version' and 'partitioning_pg_max_version' config params

- Bugfix: DB Partitioning: drop and re-create trigger if it was on another table (possibly *_previous_data)

- Bugfix: user_actions.calcRemainingDurationAndBytes, bytes is (str of) integer, not float, issue #912

- Bugfix: add BaseOnlineUsers.getOnlineUserIDsWithChargeID, use it in charge_actions.py before deleting a charge

- Bugfix: connection_usage.py: TypeError: 'NoneType' object has no attribute '__getitem__'

- Bugfix: minor bug in interface: Admin -> User Info -> Message Center -> always showed Yes

- Feature: show user attributes of (User Info's) Comment tab in Search User -> Attributes to Edit

- replace non-working isp_mapped_user_online_instances with isp_online_users_count getting from redis 

 add isp_obj.getOnlineUsersCount(), use it in isp_obj.hasOnlineUsers()

- don't send any notification (expiration/credit/birthday) to a locked user

- replace `map(None, x)` with `list(x)`

- replace `filter(lambda...)` with list comprehention

- remove `apply` with static number of arguments

- remove all the SortedList and SortedDic crap, needs to be reviewed/tested for possible typos

- replace list.sort cmp function with key function while fixing a stupid bug in user_import_factory: cmp must return 0, -1, 1

- replace all `map(lambda...` crap with list comprehension or normal for loop

- Bugfix: C_lan_acc_staging: mikrotik.py: ras_msg: rename "pppoe_service" into "service" 

 "service" is used in charge rules
 and "pppoe_service" is not used anywhere

- replace LOWERCASE_LETTERS_ON_USER_LOGIN flag with 2 new flags: LOWERCASE_USERNAME_ON_USER_LOGIN and LOWERCASE_PASSWORD_ON_USER_LOGIN

- Feature: Monthly Report Email: send connection logs of users to their email at the begining of Jalali month

- Bugfix: AbsDate: change AttributeError to a nice GeneralException

- imrovement: user panel: disconnect: change message (Persian / English) to prevent user's confusions

- Search Expired Users: show default '0 Days' for Expiration Date To, to avoid confusions

- bug fix: Copy This Group was not working, added Status to template

- C_staging: add missing perm DISALLOW ADD MULTIPLE USERS, used in user_handler.py exists in non-C branches since tag 173

- Added CHANGE INTERNET USERNAME permission and disallow changing username by default

- Feature: Add Query Wrapper for PyTable

- feature: allow admins to reset Real First Login in user info

- Cleanup: Remove PostgreSQL Storage

- add docstring for getUsersExpDateFirstLogin in user_actions.py, and remove extra newlines

- Feature: Add PyTables Log Type

- Feature: Add `md5_hexdigest` Function

- Feature: write LI log for shahkar service

- Feature: add flock

- fix interface bug in Bulk Renew Users: did not show error message

- sendNormalPasswordSMS: only check for attr_name='cell_phone'

- login_actions.py: break long lines

- send password: login_actions.py: fix style and remove uneeded lines

- security fix, add config options send_password_email and send_password_sms for web service, disable sms by default

- add Send Password feature (via EMail / SMS) as an interface config option

- disable XSS cleaning for inserted attrs too

- GenericHTTP mc provider: always save failure reason in report 

 add "failure_response_errors" comma-seperated attribute
 do addToFailed is request failed
 not tested

- GenericHTTP provider: add success_response_pattern parameter, a regex to decide if sending message was successful

- generic_http.py: replace __findEPointObjs with a simple list comprehension in receive

تگ 369 در تاریخ ۹۵/۱۱/۱۶

- BugFix: Reauthorize bytes were deducted twice from remaining bytes

- BugFix: Do not start accounting on reauthorize. Removes lots of nagging about online dic not found in error log

- Improvement: Make sure we never return a quota of 0 in case of a successful reauthorize

- Improvement: Run internal relogin with same priority as auth. Long queues might cause WS request to return while task has not been done yet

- BugFix: reauthorize_used_bytes should be used bytes from last update and not from start of session. Fixed premature QV0 while user still has credit.

- Feature: Add support for cisco_isg_max_reauthorize_mbytes and cisco_isg_max_reauthorize_minutes attributes to force reauthorize after a certain threshold

- Feature: Multiple prepaid services per account support

- BugFix: Max Rule Traffic Usage were not working correctly with multiple rules sharing one value (and custom field) and multiple instances using them

- BugFix: Multi login did not work with Multi Service rases other than ISG


تگ 368 در تاریخ ۹۵/۱۱/۱۱

- nothing changed in this tag

تگ 367 در تاریخ ۹۵/۱۱/۱۱

- Bugfix: skip users without ignore_ras


تگ 366 در تاریخ ۹۵/۱۱/۰۴

- Bugfix: #1452 Fix/Update Deprecated Links in Graph Menu

- Bugfix: #1136 Fix Name of Import File in Import User Menu


تگ 365 در تاریخ ۹۵/۱۰/۲۹

- Improvement: Failure to change sub services (ex. failed COA) should be detected. Also resync services after internal relogin to make sure we have correct services


تگ 364 در تاریخ ۹۵/۱۰/۲۹

- BugFix: Re-Onlined users might have get failed status due to left over from previous sessions

- BugFix: Radius Attributes with integer values were not working in charge rule attribute radius

- Improvement: Allow Acct-Interim-Interval to be passed to failed users

- BugFix: Some initial variables where missing in configuration file and advanced configuration

- Improvement: API cleanups for cisco isg rases


تگ 363 در تاریخ ۹۵/۱۰/۲۶

- Bugfix: cast ignore ras ids to int

- Bugfix: read min/max event time directly from defs


تگ 361 تا 362 در تاریخ ۹۵/۱۰/۱۹

- Bugfix: fix group usages of connection analysis

- Feature: Add User ID Attribute to Search User Page

C_pmacct Branch

تگ 372 تا 373 در تاریخ ۹۶/۰۳/۲۷

- Bugfix: Fix CSV Report of Web Analyser

- Bugfix: #1339 Load Invoice Main in Online Worker

- Bugfix : #1300 delete realtime request log interface 

 real time request log  deleted  from interface because the source code of this link  had been deleted before .

- Bugfix: #1339 Load Invoice Main in Online Worker

- Feature: #1223 Modify 'urldump' sniffer application to fetch 'ip_map' data directly from IBSng

- Bugfix: Fix LDAP Permissions

- Bugfix: #1278 Fix Account Not Started Counter 

 When "account not started" counter got negative value,
 IBSng web panel shown "0*" value that means negative number.
 The main reason for this issue was when a user had several sub services,
 it tried to decrease counter per each of them (when their requests received),
 while it increased counter once on authentication action. So to solve this issue,
 we handled __decreamentNotStartedCounter method which check loaded user's
 instance that must be Master to run decrease action.
 Example diagram for this issue:
  Step1: Authentication +1   << Correct
  Step2: Master         -1   << Correct
  Step3: Free-Site      -1   << Incorrect
  StepN: SubServiceN    -1   << Incorrect
  StepX: SubServiceX    -1   << Incorrect

C_mongo Branch

تگ 372 تا 374 در تاریخ ۱۳۹۶/۰۳/۲۷

- Bugfix: #1339 Load Invoice Main in Online Worker

- Bugfix : #1300 delete realtime request log interface 

 real time request log  deleted  from interface because the source code of this link  had been deleted before .

- Bugfix: #1339 Load Invoice Main in Online Worker

- Bugfix: #1278 Fix Account Not Started Counter 

 When "account not started" counter got negative value,
 IBSng web panel shown "0*" value that means negative number.
 The main reason for this issue was when a user had several sub services,
 it tried to decrease counter per each of them (when their requests received),
 while it increased counter once on authentication action. So to solve this issue,
 we handled __decreamentNotStartedCounter method which check loaded user's
 instance that must be Master to run decrease action.
 Example diagram for this issue:
  Step1: Authentication +1   << Correct
  Step2: Master         -1   << Correct
  Step3: Free-Site      -1   << Incorrect
  StepN: SubServiceN    -1   << Incorrect
  StepX: SubServiceX    -1   << Incorrect

- Bugfix: #1292 Fix Bulk Kill Action in Case of Choosing More Than One Group 

 When user chooses more than one group/isp/etc in Search User panel and
 selects bulk kill action it raises and error. It was because of
 invalid format of passing parameters.

- Improvement: #1186 Run mongo index in thread

- Improvement: #1186 Run mongo index in thread

- Bugfix: #1235 Fix Minor Invoicing DB Bug 

 Column reference to 'pi_issue_date' was ambiguous!

- Bugfix: Correct Delete Query of Redundant Partial Log Records

- Improvement: Read From Config File Whether to Write on PSQL or Not 

 Until now we read `defs.PSQL_LOG_CONNECTIONS` value to define whether
 we should write connection logs into PostgreSQL or not. With this
 commit value of `connection_log_use_psql` in `DATABASE_MONGO` section
 of IBSng config file defines our behaviour.


تگ 372 در تاریخ ۱۳۹۶/۰۱/۲۶

- Improvement: #1154 Add More Log in Case of Bulk Write Error

- Bugfix: #1201 Fix Slow Start of IBSng and Multiple Partial Log 

 It has been seen that the number of online user doesn't match the
 records count on partial data table, it will result in slow start of
 IBSng (in recover partial log phase). This commit will fix this issue
 and makes some improvement on writing partial log data.

- Improvement: Optimized partial log recovery for postgresql. Not exists queries are much more efficient for large inner queries with index

- BugFix: Internal relogin cleanup and bug fixes

- Improvement: Increased default max thresholds to decrease the load on ras and ibs

تگ 369 تا 371 در تاریخ ۱۳۹۵/۱۲/۱۰

- Improvement: Add Missing Log for MongoDB Bulk Action Failure

- Bugfix: Fix Active Leaves 

 The `getActiveLeaves` method must be called in 'MAIN' process.

- Improvement: Add Debug Log When Bulk Action of MongoDB Fails

- Bugfix: Fix Substracting User Credit Twice on Moving Partial Data 

 When we are restarting IBSng, all records of Connection Log Partial
 Data Table/Collection will be moved to the base Table/Collection,
 meanwhile used credit must be substracted from current credit of user,
 but it will be done seperately on each plugin: PostgreSQL and MongoDB,
 this will result in substracting used credit twice from user credit
 which is WRONG; this has been handled in this commit.
   
 Also we have done some performance improvement on moving collection
 records completely (bulk action) on MongoDB.

- BugFix: First reauthorize on ISG would cause lots of nagging and also leaves session dict without necessary attributes

- BugFix: Limit Port might cause error if ignored ras is not available in user attrs

- Improvement: Second username mapping should not be flushed on restart

تگ 369 در تاریخ ۱۳۹۵/۱۱/۳۰

- BugFix: We should not add internal_relogin flag to user attributes

- BugFix: Various Internal Relogin fixes

- Improvement: Make sure listen loop won't exit on an abnormal exception

- BugFix: Setting max rule traffic usage value as a float in custom fields would have break max rule traffic plugin

- Improvement: Disable Partitioning Check Contraints, this is done because of some efficiency issues

- Improvement: Add Index on `login_time` Field of Connection Log Collection

- BugFix: We should ignore negative threshold when QV is passed to cisco isg

- BugFix: A mid session reauthorize after a restart will create an instance without start accounting

- BugFix: Do not panic if charge rule is None during stop session

- BugFix: Not saving user after logout will lead to inconsistent user object

- Improvement: Handle Mongo Query Limitation in Query Generation Function

- Bugfix: #1476 Fix CSV/PDF Report of Connection Usages in MongoDB

- Bugfix: Replace <i8> Tag of XMLRPCLib to <int> For Backward Compatibility 

 According to XMLRPC specification long integer variables (INT64)
 should be handled with <i8> tag, but unfortunately this haven't been
 handled in C#, PHP, etc languages. So we are forced to use <int> tag
 with variables in range of INT32 and use <string> tag with variables
 not in specified range.

- Improvement: Cast All Elements of Details Section of Connection Log Report to String

- BugFix: Take credit limit into account when calculating remaining bytes. This fixes postpaid user problem with isg prepaid services

- Improvement: Kill max delay default value was too large. Set it to 2 hours by default

- Improvement: Make auth rate control optional and turn it off by default

- Bugfix: #1476 Fix CSV/PDF Report of Connection Log in MongoDB

- Bugfix: #1512 Fix `int` Convertion of `float` String

- Bugfix: #1503 Fix MAC Extraction From RAS of ZTE Type 

 ZTE model rases change the format of user mac address in recent
 framework update, e.g:
 old format: `78d99fd34e4b`
 new format: `78:d9:9f:d3:4e:4b`

- Bugfix: #1500 Fix Sorting and Pagination Problem of Connection Usages Report

- Bugfix: don't assign ip to sub service


تگ 368 در تاریخ ۱۳۹۵/۱۱/۲۵

- Bugfix: add missing function call on commit of periodic acc plugin

- Improvement: Few logging for radius process

- BugFix: Reauthorize bytes were deducted twice from remaining bytes

- BugFix: Fixed bad commit

- BugFix: Do not start accounting on reauthorize; Removes lots of nagging about online dic not found in error log

- Improvement: Make sure we never return a quota of 0 in case of a successful reauthorize

- Improvement: Run internal relogin with same priority as auth. Long queues might cause WS request to return while task has not been done yet

- BugFix: reauthorize_used_bytes should be used bytes from last update and not from start of session. Fixed premature QV0 while user still has credit.

- Bugfix: skip users without ignore_ras

- Feature: Add support for cisco_isg_max_reauthorize_mbytes and cisco_isg_max_reauthorize_minutes attributes to force reauthorize after a certain threshold

- Feature: Multiple prepaid services per account support

- BugFix: Max Rule Traffic Usage were not working correctly with multiple rules sharing one value (and custom field) and multiple instances using them

- BugFix: Onlines table in single user info page were became messed up after first ajax based refresh

- BugFix: Multi login did not work with Multi Service rases other than ISG

- Bugfix: #1452 Fix/Update Deprecated Links in Graph Menu 

 The following links are deprecated and removed:
 All Realtime Graph
 Internet Realtime Graph
 Voip Realtime Graph
 BW Realtime Graph
 Also updated modified links

- Improvement: Failure to change sub services (ex. failed COA) should be detected. Also resync services after internal relogin to make sure we have correct services

- Bugfix: #1136 Fix Name of Import File in Import User Menu

- BugFix: Re-Onlined users might have get failed status due to left over from previous sessions

- BugFix: Radius Attributes with integer values were not working in charge rule attribute radius

- Improvement: Allow Acct-Interim-Interval to be passed to failed users

- Improvement: Exception handling for internal relogin

- Bugfix: Incorrect call to getUserAttrs caused exception

- BugFix: Some initial variables where missing in configuration file and advanced configuration

- BugFix: Undefined session raised unnecessary exception

- Improvement: API cleanups for cisco isg rases

- Bugfix: cast ignore ras ids to int

- Bugfix: read min/max event time directly from defs

- Bugfix: fix group usages of connection analysis

- Feature: Add User ID Attribute to Search User Page

- Feature: optionally do autoRenew on abs expired user

- Bugfix: log deposit_type in deposit change

- Cleanup: clean md5 response


تگ 367 در تاریخ ۱۳۹۵/۱۱/۱۳

- Bugfix: #1449 Fix `users.group_id` Bug

- Feature: #1478 Add `bson.int64.Int64` Marshaling Support to XMLRPCLib

- Bugfix: #1465 Fix Remaining Credit Feild of Connection Log Report 

 Some older records (or first record) of user connection log may not
 have before credit field or has the value `None`, which yields to
 raising an exception when calculating the value of `remaining_credit`
 field [`remaining_credit = before_credit - credit_used`].

- Bugfix: #1449 Fix Ambiguous Fields `user_id` and `group_id` in Management Summary Report 

 After adding two columns `user_id` and `group_id` to `connection_log`
 table it created a conflict with `users` table on in management
 summary queries; in this fix we change SQL queries to exactly specify
 which fields we mean by `user_id` and `group_id`: `users.user_id`,
 `users.group_id`, etc.


تگ 366 در تاریخ ۱۳۹۵/۱۱/۰۹

- Feature: Add `getConnectionUsages` Method to API 

 The old methods of connection usages report (`getCreditUsages`,
 `getDurationUsages` and `getInOutUsages`) won't support ISG feature:
 `sub_service_name`, `sub_service_charging` and `sub_service_qos`.
 So we added a new method called `getConnectionUsages` which covers
 ISG features.

- Bugfix: #1460 Fix Connection Usage Report


تگ 365 در تاریخ ۱۳۹۵/۱۱/۰۵

Improvement: Second username mapping should not be flushed on restart

BugFix: Re-Onlined users might have get failed status due to left over from previous sessions

Improvement: Allow Acct-Interim-Interval to be passed to failed users

Bugfix: Incorrect call to getUserAttrs method of API caused exception

BugFix: Some initial variables where missing in configuration file and advanced configuration

BugFix: Undefined session raised unnecessary exception

Improvement: API cleanups for cisco isg rases

C_diameter Branch

تگ 401 تا 402 در تاریخ ۱۳۹۶/۰۷/۲۶

BugFix: Previous usage for maximum rule traffic usage with a custom field used in multiple rules were not calculated correctly

BugFix: applicableUntil of max rule traffic usage plugin with same custom field as value of multiple rules were not working correctly, leading to incorrect usage calculations

Improvement: Updated sample configurations

BugFix: Outgoing radius packets initiated from ws processes were not logged

Bugfix: Fix a Change in Previous Commit

Bugfix: #1682 Remove `addToBlackList` Cause We Don't Use It Any More 

This method belonged to `UserPool` class which was removed in
master/slave feature.

Improvement: Allow passing extra arguments to canStay

Improvement: Send ras msg in internetReauthorize so charge plugin can use reauthorize_use_bytes in calculations

Improvement: Redirect user to user info after credit change using GET instead of Post, so user info page will be refreshable

Bugfix: #1663 Removing Leaves Should be Ran in Online Worker

Improvement: Make less query while loading user to optimize performance

Feature: #1645 add a white list for referrers to config 

add a white list to section "INTERFACE" inside IBSng.conf.default

Improvement: #1550 Change Date Limit from 10 to 15 Years

Improvement: Allow abs exp date up to 15 years

BugFix: Empty cisco_isg_failed_service was not supported with FAILED_USER_L4 in cisco_isg_failed_method

BugFix: Force Auto Recharge on zero credit was not working if deposit was added while user is offline

BugFix: Max Rule Time Usage previous value were not shown in edit template

BugFix: A service rule without negative credit while user has credit limit less than zero would not trigger a reauthorize

NewFeature: auto_recharge_force_on_zero_credit Feature to perform auto recharge even when credit limit is fewer than zero and negative credit has been set

BugFix: Radius Queue Statistics were not working on multiple machines

Improvement: FAILED_USER_L4 will always send services cisco_isg_failed_service separated by comma and appended by __Unlimited

BugFix: Radius packets were missed because of typo

BugFix: Partial log retry on lock were incorrect causing massive retry of pointless partial logs

BugFix: Normal stop were not working because of typo

BugFix: Make sure we have enough user pool capacity and we will not leak loaded user objects

BugFix: Non-priority queue is not multi login friendly causes locks being hold while user is in queue

Improvement: Allocate db pools in 3 batches to prevent everyone allocate a handle on start situation

BugFix: Online locks should always be distributed as user has no native process

BugFix: Server name should not be in online_users table anymore

BugFix: Worker Queue was malfunctioning leading to incorrect counter values

BugFix: Various new mm fixes

BugFix: Internal Relogin were not working

Improvement: Throttle getting distributed events if worker queue are high

BugFix: Blocking mode of radius job polling caused reply loop not to work. Fixed by running another thread for receive of replies.

Improvement: Use ippools directly in RADIUS_WORKER processes to prevent expensive mainEvaluate calls

BugFix: Throttled job polling loop did not wait for job queues to become lower than threshold and just waited once

BugFix: Adding periodic accounting while user is online would cause user info page not to work

Improvement: Reversed the job acquire mechasim from radius workers pushing to online workers into online workers pull from radius workers

BugFix: partial log event time was incorrect

Improvement: Change user pool size based on process. Online workers does not need to keep loaded users anymore

BugFix: Idle users were not compatible with new multi master

Enhancement: Remove Recalc In Queue Detection as it's not applicable anymore

BugFix: Can Stay Runner were not compatible with new multi master

NewFeature: Redis Backed, Shared State IPPools.

BugFix: Clear user were not working with new multi master

BugFix: Internal Relogin were not working with new multi master

NewFeature: Distributed Event and Delayed Queue Feature

First Working version of stateless online workers

تگ 400 تا 401 در تاریخ ۱۳۹۶/۰۷/۱۵

Bugfix: Fix Sanitization to Ignore Some Strings 

By applying sanitization on web request, following to character are
recognized as html tags and so will be ignored (deleted) from request.

Improvement: Do not kill service instances as change service will take care of that

NewFeature: If Not Sub Service Charge Rule Attribute to activate a service only when another service is not activated

Feature :#1647 add step for ippool percentage in config 

step number added for ippool percentage in config file in INTERFACE
section with interface_ippool_step TAG

تگ 398 تا 400 در تاریخ ۱۳۹۶/۰۷/۰۳

Support Request: #1555 Add User Import Log

Feature: Port auth_event Plugin on C_diameter Branch

Feature: #1460 Send User Events on Message Broker

Improvement: Stop Clearing Terminal on IBSng Startup

Bugfix: #1634 fix connection log report problem 

this problem caused because of new implementation for xss prevention
fixed the place of the code.

Improvement: #1617 Add PubSub to Invoice Profile

Feature: #1553 Add PubSub to Custom Field 

This commit is copy/paste of PR No. 23 which has been done by Morteza
Nourelahi Alamdari on development branch. Because this is a priority
for us in stable branch we create this commit.

Bugfix: #1525 bulk action not found error fix 

it isn't fixed completely yet since ws_workers are more than 1, and
bulk action instance may not have action_id in every ws_worker hence
we face this error. so we sated action_id to $_SESSION so we will
keep it in front side for getting info about it.

Improvement: #1618 Correct Space between Codes

Feature: #1239 Add `online_payment.getAvailableGatewayTypes` Method 

This method have been copy/paste from staging branch because it's been
needed for Wispot project and we can't wait for merging staging branch
to stable one.

Feature: #1611 Add Forgot Password to User Web Panel 

Actually this feature has been implemented before in staging branch by
Saeed Rasooli. Since we are not ready now to merge our staging branch
to stable one, so we have some how copy/paste the changes (with some
cleanup) from staging branch to answer customer request.

Improvement: #1618 Return admin_id in addNewAdmin Method

Feature: #1572 fix xss security problem and sql injection 

xss csrf and auto complete in form have been fixed .

Bugfix: #1515 fix all online graph's time line in user and admin panel 

all the time line issues in both admin panel and user panel for real time graphs have been fixed for branches without mongoDB.

Improvement: #1550 Validate Absolute Expiration Date Input 

Check the epoch time that it won't pass 10 years from now.

Feature: #1146 Add New Permission "CHANGE USER LIMIT MAC" 

From now on, only admins with specified permission can modify the
limit mac attribute of users. This permission has "CHANGE USER
ATTRIBUTES" permission as it's parent.


تگ 394 تا 398 در تاریخ ۱۳۹۶/۰۶/۰۶

Improvement: Online state keeper refactor and code cleanups to improve performance and correctiveness

Improvement: Ensure we don't leak redis connections in rare cases of pipelines

Cleanup: 0 does not need to be passed to price modifier

Improvement: Made online worker concurrency configurable which is count of worker threads in each online worker. The actual concurrency would be twice this number because threads sending requests to user workers and threads receiving results from user workers are separated

Improvement: Change the order which db handles are used to enable idle checking. It was a FIFO queue and now its LIFO.

NewFeature: Rebin Integration Support

Improvement: Implemented fast shutdown specifically to flush online user count queue before force killing.

Cleanup: Cleanup mpi adaptor codes

Enhancement: Refactor dbpool to use less connections to db. Also it releases db handles if they are not used more than defs.DB_POOL_IDLE_RELEASE_TIME which is 15 minutes by default

BugFix: Not flushing charge rule cache on reload might cause problems with state changes

BugFix: Fixed double caching loaded user in user_obj causes loaded user not to be reloaded correctly

BugFix: Multi Master and LIMIT LOGIN ADDR permissions were not compatible

BugFix: Proxy of requests with auth_pass did not work

Improvement: Missing instance during killAllInstances operation is normal and should not be logged into error log

BugFix: Negative CPMB caused credit notification to go into a very large loop

BugFix: Fixed typo in error message

Improvement: Kill and Clear must be run on owner server

BugFix: Fast Start was not working with license

BugFix: Fix kill was not working due to change in ras redis attributes

BugFix: We should raise Exception if we can not move a user

BugFix: Background and font color of notification was the same making them unreadable

BugFix: Template closing tag was incorrect

BugFix: Background and font color of notification was the same making them unreadable

BugFix: Template closing tag was incorrect

Improvement: Pass user_obj to user_workers through queues instead of redis to improve performance

NewFeature: Ability to run on multiple machines and fast restart

Bugfix: #1520 Fix Connection Log Access in User Panel 

When a request calls the "report.getConnection" method, requester will
be a Normal User which don't have "hasPermOrGOD" method. This have
been handled in this commit.

BugFix: Enhancements in online object to prevent user object corruption

Improvement: Run two threads for handling each user worker instead of one to improve performance

Improvement: shutdown users enhancement on clear only scenario

BugFix: Fix Extra charge was applying with a large delay due to slow mid night events

BugFix: Daily event start debug log were never written

Improvement: Refactor successful auth boolean

BugFix: Refactor multilogin plugin to not keep multilogin_not_allowed in plugin state and reduce memory usage

BugFix: Fixed typo causing user worker feeder loop to slow down

Improvement: Improve thread latency

BugFix: Some redis dbs were not flushed on startup

Improvement: Do not use multiple dbs for ras data. Redis people discourage using multiple dbs

Improvement: Process requests of RADIUS_WORKER in 4 threads instead of 1 to improve performance

Improvement: Add autosync_on_change_service option to pcef rases to resync instead of change service 

This fixes douran bug which does not understand change services

Improvement: Add auth by caller id support to huawei ras


تگ 385 تا 394 در تاریخ ۱۳۹۶/۰۵/۲۳

Improvement: Allow defining two other redis instances for radius and online_user_storage for redis clustering

BugFix: 3GPP location info is in binary format and should be converted

Feature: Add "FULL ACCESS CONNECTION LOGS" Permission 

 Admins with this permission can set all filters of connection
 log/usage report (like a GOD admin). This permission has "SEE
 CONNECTION LOGS" dependency.

Bugfix: Fix Result Per Page Limit of Connection Log 

 This problem occured after limiting users on accessing certain filters
 in connection log report page, where setting result per page parameter
 has no effect on output result.

Feature: #1477 fix access for non GOD users to connection usage

Feature: #1477 add control permission for non GOD users to connection usage 

 remove advanced and group tabs and remove isp from conditions tab.
 and the maximum range of login_from , login_to is 1 month now.

BugFix: runDebugCode were not working in WS_WORKERS

BugFix: extractRoutes paniced in case ras set routes in prefix bit

NewFeature: Allow having multiple webservice processes

BugFix: Typo in expirePIQuery resulted in no PI expiration

Bugfix: #1412 Change Back Using Event Time Instead of Remaining Time 

 In `canStayOnlineResult` when we intend to use `newRemainingTime` we
 have to make sure about exact value of this parameter (no estimation
 here), otherwise we have to use `newNextEventTime`.

Bugfix: #1412 Using `get` for Fetching Threshold Values

Bugfix: #1412 Fix Using of Thresholds in Can Stay Online Result 

 If can stay online result object contains `remaining_bytes` or
 `remaining_time` we will fill our `session_dict` with
 `threshold_bytes` and `threshold_secs` respectively.
   
 Now if an alive packet arrives and one of these thresholds meet, we
 run `canStayOnline` methods of user plugins (by force) to recalculate
 the parameters. This will fix Periodic Accounting and Notification
 Profile plugins functionality.
  
 Note: We have error threshold of twice the interval between alive
 packets, meaning, we may run `canStayOnline` methods late by specified
 time.

Improvement: Increase udp memory buffers to improve radius server performance

Improvement: Use blocking recvfrom instead of select as now we have multiple processes recvfrom same socket

Bugfix: Add `routes` Parameter to Cisco PCEF `provisionRequest` Method

NewFeature: Implemented pretty start using blessings to emphasize important events on startup

BugFix: Diameter process should not process any radius packets

BugFix: Fixed duplicate packets were not working

Improvement: Moved RADIUS_AUTH and RADIUS_ACCT into RADIUS_WORKER for better scalability

BugFix: Fixed douran provisioning did not support routes

BugFix: Framed-Route passing to pcef were not working on huawei rases

BugFix: Fixed typo which caused Traffic Periodic Accounting without cpmb to not work

BugFix: AppliedChargeRule.getChargeRule should return None if charge rule is no longer available

Bugfix: Add Two Attributes to Radius Dict; `byte` and `ipv6addr`

Improvement: Some Changes for Non GOD Report of Connection Log 

 Also Add new attribute types of 3GPP.

Feature: #1447 add permissions for non God admin in connection log 

 added some permission for non god admins witch non god admins wont see some tabs in connection log page and range of login time will be at most  one month .

BugFix: CPMB input validation was incorrect

Improvement: API change bw were not working in C_diameter based branches

NewFeature: 3GPP attribute support in huawei ras

BugFix: Fixed typo in sendCOA caused log not to be written in console for COAs

BugFix: Append a * to custom field attribute names to indicate its a custom field and prevent clash of request names with conditions

Improvement: Allow setting negative cpmb as a mean for services to reduce from master traffic

Bugfix: #1455 Change Priority of User Rad Attr Over Charge Rad Attr 

 In some cases (such as "Filter-Id" attribute) Rases will only consider
 the latest attribute in case two (or more) attributes have same
 keys. Because user radius attributes have priority over charge radius
 attribute by setting it's priority to 7, it will run after charge plugin
 (which it's priority is 6) so the considered attribute value will be
 the user's one.

Feature: #1397 Remove Admin Condition From User Audit Log 

 We replaced Admin condition in User Audit Log page with User ISP
 condition where the records will be filtered based on accessable
 user(s) in ISP instead of accessable Admin(s) of ISP.

NewFeature: Support for routes in pcef user awareness

Feature: #1462 add failed online reason to user information page 

 if there was a reason for failed online it will be shown on user information page .

Improvement: Enhanced traffic periodic accounting to use can stay remaining bytes and internal reauthorize

BugFix: failed_login_assign_ip was not applied to routeip and netmask plugins

BugFix: IPHost-Addr attribute were changed in new huawei dictionary

BugFix: An error during periodic credit change checkAll might cause skipping of change credit for some users

Improvement: Script to reload radius dictionary. Should be run in all radius workers

BugFix: Huawei DNS attributes were changed in new dictionary

BugFix: Duplicate Mac/Port did not work after last change

Improvement: A handy script to kill all failed users

BugFix: Failed users does not need internal relogin as killIfFailed would take care of that

BugFix: getOnlineUsersIterItems was returning user_id as string instead of integer

BugFix: Fix typo in table name

Improvement: New client script to clear all acct not started users

BugFix: Huawei ras was not compatible with new dictionary file

Improvement: Updated huawei dictionary

Bugfix: #1417 Minor Cleanup

Bugfix: #1417 Fix Load Balancing Functionality of IPPool Plugin 

 The `ippool_main.getLoader().getIPpoolByID` will return either an
 "IPPool" or "LoadBalancingIPPool" object. In later case the object has
 no `getContainer` method. Instead we use `setIPInPacket` method which
 is available in both objects and has the same functionality.

Cleanup: Removed ONLINE_CHECK_MULTI_LOGIN from advanced config as it was duplicate functionality with multi login plugin

BugFix: Failed logins should not be counted into multi login. Now allow up to 3 failed logins in addition to normal master instances.

Feature: #1317 Add ISP Filter to `getListOfCharges` Charge Action

Feature: #1317 Add ISP Filter to `getListOfGroups` Group Action

BugFix: IP Pool plugin was raising None if ip is not available in ippool anymore

Improvement: Can Stay Result now have 4 variables 

 Remaining Time: Charging and Authorization remaining Time
 Next Event Time: Used by plugins need to set a next event
 Improtatnt Event Time: This overrides MIN_USER_EVENT_TIME
 Remaining Bytes: Bytes remaining until next event

Improvement: Internet Reauthorizes which triggered auto recharge were rejected incorrectly

BugFix: Charge.getRuleWithID should return None if rule id does not exists

NewFeature: diameter_prevent_cctime_avalanche attribute in diameter ras controls if cc-time should be distributed across half of a update accounting time

NewFeature: Persistent charge rule cache to use a LFU cache for keeping applied charge rule data

BugFix: Diameter terminate would reset session dict by mistake

BugFix: Diameter response time can be higher than usual radius authentication

BugFix: Packet storage should not be opened in online worker due to performance penalty

NewFeature: Added support to specify unix socket or ip address of syslog through syslog_destination in config file

Bugfix: Fix Minor Bug

BugFix: Kill failed user if deposit has been changed in hope of auto renew or auto recharge

Improvement: added REQUIRE_INTERNAL_RELOGIN flag to AttributeHandler to control if internal relogin is required for updating the specific attribute. This will reduce number of internal relogins required

BugFix: IBSErrors are mostly login related and should not be logged in error.log

BugFix: Failed and Idle login automatic detection on re-online were not working

Cleanup: Removed unused DEBUG_ERRORS, DEBUG_WARNINGS, DEBUG_THREADS from defs and logging

Cleanup: Removed duplicate defs entries

Improvement: Allow changing including of seconds in date through advanced config

BugFix: _handlePacketResponse signature was changed while some usages were missing

Bugfix :#1436 change QOS defs propertis to accept string input

BugFix: Fix small memory and thread leak problem in thread wrappers

BugFix: Douran ras was using charging name to be passed as rating group. It should be extracted from sub service name

NewFeature: Reject cache configuration has been moved to ras attributes. [REJECT CACHE] Configuration section is now deprecated.

NewFeature: Diameter translated packets are now written into DIAMETER/diameter_translation.log instead of filling up RADIUS/auth.log

Improvement: Diameter packets will use new packet codes DiameterCCRInit,DiameterCCRUpdate,DiameterCCRTerm,DiameterCCASuccess,DiameterCCAFailure instead of radius ones

BugFix: True or False should not be used in defs variables. 1 or 0 should be used instead

Bugfix: Replace `__bool__` With `__nonzero__` 

 `__bool__` data model is replacement of `__nonzero__` in python 3.x;
 since we're still on python 2.x we have to still use `__nonzero__`

NewFeature: Support for setting cisco_isg_failed_method attribute of cisco isg ppp ras to FAILED_USER_L4 which causes failed user to have normal ip assignments

Bugfix :#1424 real time bw graph timeline problem 

 time line problem solved using utc time guide.

Improvement: don't log wrong password

Improvement: Show session id (unique_id_value in fact) in connection log report

Improvement: Show seconds by default in all reports

BugFix: if pkt was checking attributes length while it should be always true

Improvement: Log storage address in radius logs for debugging purpose

Improvement: Checkonline was running too fast causing Late Packets during it's run

Improvement: Better logging for packet storage

BugFix: Prevent double slashes for shared storage file paths

BugFix: threadpool terminate and close methods were not working

Improvement: Log ibs request id into radius log to make debugging easier

BugFix: In some situations, cisco pcef session did not stop after master session has been disconnected. Make sure the session is cleaned up by first sending pod if session pcef dict is not available and second optionally send a pod on each start packet

BugFix: Fixed a race condition that might cause packet storage files not to be deleted

Bugfix: #1367 search_expired_user.php check_all_users problem 

 multi select problem for users has been handled.

BugFix: Policy worker server log were not filtered

BugFix: snmp agent were not working due to xmlrpc limitations. Converted client to use jsonrpc instead

Cleanup: Refactor threadpool to be more efficient and simple. Also there would be no need to tune number of threads for different sizes of system as threadpool would grow automatically

Improvement: Do not log unnecessary attributes in connection log

Cleanup: Do not log soap handler registeration if debugging is off

Cleanup: Removed not migrated ras folder

BugFix: Exception Logs were written twice

BugFix: IP Conflict happens during re-online because conflicted session did not get kicked out

Improvement: Include seconds in date representation

Improvement: Timedelta representation for kill time is ugly. Use default date format instead


تگ 381 تا 385 در تاریخ ۱۳۹۶/۰۴/۱۵

- Bugfix: #1408 Port call_id_blacklist plugin to C branch

- BugFix: Moved SOAP HTTP Server from builtin python library into a local library and swap select.select to poll.epoll

- BugFix: Stop priority should be higher

- BugFix: Fixed internal relogin might raise (harmless) error if called multiple times

- BugFix: Cisco VPDN was nagging incorrectly if user does not have in ras bw

- BugFix: Not finding session dict during session start is not an error and should not be reported as an error

- BugFix: Maximum reauthorize quotas did not apply if effective charge rule does not have CPMB

- BugFix: Cisco PCEF stop awareness should remove session from activated sessions

- BugFix: pass_remaining_volume and pass_remaining_duration ras attributes were not applied in cisco isg races

- BugFix: Internal Relogin would cause ip conflict if ippool is defined in user or charge level

- BugFix: Internal Relogin kill reason was lost due to not saving user state

- BugFix: Send service resync requests after internal_relogin with a delay to make sure all instances has been relogged in before resync

- Improvement: Clean up imports so unnecessary components would not be initialized in listener processes

- BugFix: User Worker Queue did not run requests in order which their arrived even if they had equivalent priority

- BugFix: ID generation was not working correctly for multiple processes. Fixed by separating port range of each radius worker

- Improvement: Cache time series dynamic register statistics to decrease redis hgetall calls

- Fix: Show Exact Value for ISP's on Online Users

- Feature : #1361 change the sub service qos name to limited drop down 

 change  the sub service qos name to limited drop down options .

- Feature : #1360 edit charge priority from 1-10 to 1-50 

 edit charge priority to 1-50 and it's backend error and validation rules.


تگ 377 تا 381 در تاریخ ۱۳۹۶/۰۳/۲۷

- NewFeature: Rates for diameter request and replies and also exposing them via snmp

- Bugfix: #1339 Load Invoice Main in Online Worker

- BugFix: Make ibs init script lsb compatible

- NewFeature: Script to gracefully reload charges

- Bugfix : #1300 delete realtime request log interface 

 real time request log  deleted  from interface because the source code of this link  had been deleted before .

- Improvement: Kill support of PCEF Master session

- BugFix: Re-online might mess up with Cisco PCEF Sessions

- Improvement: A check script for cisco pcef session validation

- Improvement: Log if reauthorize has changed any service

- Bugfix: #1339 Load Invoice Main in Online Worker

- Improvement: Increased isg session ttl to make sure no session is being lost

- Improvement: Updated SNMP bridge to support more OIDs and be compatible with C branch

- BugFix: Cisco PCEF stop awareness did not work sometimes due to incorrect session id

- Bugfix: Session dictionary might be lost if the time between master session and starting of traffic is more than 4 update accountings. Make is 24 hours instead

- Improvement: Notify admin about starting and finishing partial and second user mapping

- Bugfix: Re-Online did not create cisco pcef session correctly

- Bugfix: Proxy-State were not correct for reject cache responses. It should be exactly the same as request value

- BugFix: First snapshot online loop should not happen in start critical path

- Improvement: New roundrobin policy queues to prevent a unreachable ras to slow policy changes of other rases

- BugFix: 0 in/out would returned if a diameter ras does not receive tariff change reauthorize

- Bugfix: #1278 Fix Account Not Started Counter 

 When "account not started" counter got negative value,
 IBSng web panel shown "0*" value that means negative number.
 The main reason for this issue was when a user had several sub services,
 it tried to decrease counter per each of them (when their requests received),
 while it increased counter once on authentication action. So to solve this issue,
 we handled __decreamentNotStartedCounter method which check loaded user's
 instance that must be Master to run decrease action.
 Example diagram for this issue:
  Step1: Authentication +1   << Correct
  Step2: Master         -1   << Correct
  Step3: Free-Site      -1   << Incorrect
  StepN: SubServiceN    -1   << Incorrect
  StepX: SubServiceX    -1   << Incorrect

- BugFix: Make check online less intrusive by increasing sleep time

- BugFix: Re-online packets did not fill pcef master session id causing pcef stop not to work for re-onlined sessions

- BugFix: Idle threshold were not working correctly with sub services, causing users to be detected idle incorrectly

- BugFix: Better handling of isg session dict to fix pcef stop requests not being sent

- Improvement: Allow kill requests on all policy workers

- Improvement: Suppress unnecessary errors when receiving reauthorize

- BugFix: reloadUser without wait for lock caused infinite loop

- BugFix: Online User Storage must be flushed during IBSng start up

- BugFix: Credit Limit was displayed incorrectly

- BugFix: Redis might incorrectly reset pubsub in busy situations

- BugFix: Mutable object should not be passed as default argument

- Cleanup: Cleaned up check online method

- BugFix: Fixed reloadUser might incorrectly run while lock has not been acquired

- Bugfix: #1292 Fix Bulk Kill Action in Case of Choosing More Than One Group 

 When user chooses more than one group/isp/etc in Search User panel and
 selects bulk kill action it raises and error. It was because of
 invalid format of passing parameters.

- Bugfix: #1235 Fix Minor Invoicing DB Bug 

 Column reference to 'pi_issue_date' was ambiguous!


تگ 372 تا 377 در تاریخ ۱۳۹۶/۰۲/۱۹

- BugFix: mp observer change in isp loader caused isp states to reset

- BugFix: unload/load instead of reload causes race condition where isp object is no longer available

- BugFix: Reporting Reason was not parsed correctly

- Improvement: Added two more timers to radius packets

- BugFix: Online user storage were using incorrect redis db

- BugFix: Reporting Reason is a sub attribute of Used-Service-Unit.

- Improvement: Improve internet reauthorize response time by increasing the priority of reauthorize initiated recalc next event queries

- BugFix: Fixed Typo which cause isp deposit role back not to work correctly

- BugFix: Consume ISP deposit was broken after Internal Relogin. Fix to always reload ISP and log after consuming deposit.

- BugFix: Fixed diameter update messages might not be replied by better error handling of internetReAuthorizeResult

- BugFix: Make sure we always call canstay callbacks. Also use canstay callbacks instead of specific purporse reauthorize in progress mechanism

- BugFix: Do not send Failover diameter attribute as not all PGW supports it

- BugFix: Diameter ASR and RAR were not working after switching to policy workers scheme

- BugFix: Webservice port selection for RADIUS and DIAMETER processes were messed up

- BugFix: Fix memory leak of web service process by replacing cyclic-reference-creator collections.OrderedDict with odict.odict in kairos library

- BugFix: __internetReauthorizeCheckForIncorrectSubService was not working due to typo

- Improvement: Added Allocation-Retention-Priority to diameter dictionary

- Improvement: Rename KILL Process to POLICY_WORKER and allow multiple policy worker to be run simultaneously

- Improvement: Sandvine ras compatibility improvement

- Improvement: Support for separated In/Out bytes in diameter requests

- Improvement: Support more 3gpp attributes in CCRs

- NewFeature: Log watchdogs, init_and_terminates, updates, asr_and_rar diameter requests and replies in different files under DIAMETER folder

- Bugfix: #1228 Fix Max Time Usage Charge From Custom Field 

 Resolving 'max_rule_time_usage' charg rule to be read from user
 custome field (e.g: 'F:time') haven't been handled.

- BugFix: Duplicate detector was not caching reply packets and never replay them

- BugFix: RADIUS,ACCT,DIAMETER request id generation was not correct

- BugFix: Reject cache leaked memory in redis, with non-expiring keys

- BugFix: Reject cache might show incorrect macs in user interface

- NewFeature: Sandvine PCEF Support

- Improvement: Diameter Server Enhancements to comply with 3GPP 32.299 v.12 diameter charging application

- Bugfix: #1215 Fix Timezone Issue in Online Graph

- BugFix: Turned off sub service check by default

- NewFeature: New features for sub services starting and checking 

 *Sub services has now two new boolean options, auto_start and auto_check
 auto_start: Should this sub service auto started when subscriber master session brought up
 auto_check: Should this sub service checked during can stay and ensure it's available.
   
 *Service Auto Check is a new mechanism which compares current subscriber services with applicable ones.
 If there's a difference, it will resync services after a guard time. Guard time prevents resyncing when sub services in queue to start.

- Improvement: Log next event and kill dict in can stay event of log console

- Improvement: Few Logging for diameter translator

- BugFix: Cleanup diameter server connection after connection is closed

- Improvement: Use a TTL based dictionary for diameter packets to ensure we dont eat up all memory due to a problem in packet processing.

- BugFix: Fixed InternetReauthorize were not working correctly because of bad merge

- BugFix: SOAP Server is only available in WS process

- Improvement: Increased default threshold megabytes and update interval for better performance

- Improvement: Decreased number of threads for protocol listeners

- BugFix: Cisco ISG reauthorize threshold ras attributes were missing

- BugFix: Cisco ISG IPAccountLogon packet detection was not accurate

- Improvement: Run Radius Auth, Radius Acct and Diameter in separate processes to increase scalability

- BugFix: Cisco PCEF Stop did not work due to bad session id

- BugFix: Added missing imports due to bad merge

- BugFix: Internal relogin cleanup and bug fixes

- Improvement: Increased default max thresholds to decrease the load on ras and ibs

تگ 372 در تاریخ ۱۳۹۵/۱۲/۲۲

- Improvement: Decreased number of threads for protocol listeners

- BugFix: Cisco ISG reauthorize threshold ras attributes were missing

- BugFix: Cisco ISG IPAccountLogon packet detection was not accurate

- Improvement: Run Radius Auth, Radius Acct and Diameter in separate processes to increase scalability

- BugFix: Cisco PCEF Stop did not work due to bad session id

- BugFix: Internal relogin cleanup and bug fixes

- Improvement: Increased default max thresholds to decrease the load on ras and ibs

تگ 364 تا 371 در تاریخ ۱۳۹۵/۱۲/۱۰

- Bugfix: Fix Active Leaves 

 The `getActiveLeaves` method must be called in 'MAIN' process.


تگ 364 در تاریخ ۱۳۹۵/۱۲/۰۷

- Bugfix: Add Missing Imports to Radius Module

تگ 363 در تاریخ ۱۳۹۵/۱۲/۰۴

- BugFix: Internal relogin cleanup and bug fixes

- Improvement: Increased default max thresholds to decrease the load on ras and ibs

تگ 355 تا 362 در تاریخ ۱۳۹۵/۱۲/۰۲

- BugFix: Some ASRs does not allow logoff request without username

- BugFix: Various Internal Relogin fixes

- Improvement: Make sure listen loop won't exit on an abnormal exception

- BugFix: Setting max rule traffic usage value as a float in custom fields would have break max rule traffic plugin

- BugFix: Using same ID generator for radius and diameter process will cause concurrent access to generator

-Improvement: Increased number of threads in radius processes to 200. PCEF rases might establish lots of connections which run the process out of threads

- Improvement: Disable Partitioning Check Contraint, This is done because of some efficiency issues

- BugFix: We should ignore negative threshold when QV is passed to cisco isg

- BugFix: A mid session reauthorize after a restart will create an instance without start accounting

- BugFix: Not saving user after logout will lead to inconsistent user object

- BugFix: Take credit limit into account when calculating remaining bytes. This fixes postpaid user problem with isg prepaid services

- Improvement: Kill max delay default value was too large. Set it to 2 hours by default

- Improvement: Make auth rate control optional and turn it off by default

- Bugfix: #1512 Fix `int` Convertion of `float` String

- Bugfix: #1449 Fix `users.group_id` Bug 

 Fixed `getDBColName` in a way which returns the part after the dot,
 e.g 'usersgroup_id' => 'group_id'

- Bugfix: #1449 Fix Ambiguous Fields `user_id` and `group_id` in Management Summary Report 

 After adding two columns `user_id` and `group_id` to `connection_log`
 table it created a conflict with `users` table on in management
 summary queries; in this fix we change SQL queries to exactly specify
 which fields we mean by `user_id` and `group_id`: `users.user_id`,
 `users.group_id`, etc.

- Bugfix: #1503 Fix MAC Extraction From RAS of ZTE Type 

 ZTE model rases change the format of user mac address in recent
 framework update, e.g:
 old format: `78d99fd34e4b`
 new format: `78:d9:9f:d3:4e:4b`

- Improvement: Few logging for radius process

- BugFix: Reauthorize bytes were deducted twice from remaining bytes

- BugFix: Do not start accounting on reauthorize. Removes lots of nagging about online dic not found in error log

- Improvement: Make sure we never return a quota of 0 in case of a successful reauthorize

- Improvement: Run internal relogin with same priority as auth. Long queues might cause WS request to return while task has not been done yet

- BugFix: reauthorize_used_bytes should be used bytes from last update and not from start of session. Fixed premature QV0 while user still has credit.

- Bugfix: don't assign ip to sub service

- Feature: Jetsib PGW Support

- Bugfix: skip users without ignore_ras

- Feature: Add support for cisco_isg_max_reauthorize_mbytes and cisco_isg_max_reauthorize_minutes attributes to force reauthorize after a certain threshold

- Feature: Multiple prepaid services per account support

- BugFix: Max Rule Traffic Usage were not working correctly with multiple rules sharing one value (and custom field) and multiple instances using them

- BugFix: Onlines table in single user info page were became messed up after first ajax based refresh

- BugFix: Multi login did not work with Multi Service rases other than ISG

- Bugfix: #1452 Fix/Update Deprecated Links in Graph Menu 

 The following links are deprecated and removed:
 All Realtime Graph
 Internet Realtime Graph
 Voip Realtime Graph
 BW Realtime Graph
 Also updated modified links

- BugFix: Limit Port might cause error if ignored ras is not available in user attrs

- BugFix: PCEF stop provisioning was not sent

- Improvement: Second username mapping should not be flushed on restart

- Improvement: Failure to change sub services (ex. failed COA) should be detected. Also resync services after internal relogin to make sure we have correct services

- Bugfix: #1136 Fix Name of Import File in Import User Menu 

 Note: after changing the log process to pass the logging mechanism to
 python syslog module, user_import logs won't be written to a local
 file which will cause to "View Import Log" link on "Import User" page
 after uploading CSV file won't work. this can be fixed by adding new
 log type (aside `QueueLogger` and `DirectLogger`) to IBS logger
 module, but this is not a priority now; look at the comments on
 changed files for more info.

- BugFix: Re-Onlined users might have get failed status due to left over from previous sessions

- BugFix: Radius Attributes with integer values were not working in charge rule attribute radius

- Improvement: Allow Acct-Interim-Interval to be passed to failed users

- Improvement: Exception handling for internal relogin

- Bugfix: Incorrect call to getUserAttrs caused exception

- BugFix: Some initial variables where missing in configuration file and advanced configuration

- Improvement: API cleanups for cisco isg rases

- BugFix: Undefined session raised unnecessary exception

- BugFix: Some initial variables where missing in configuration file and advanced configuration

- Feature: Cisco ISG IPoE as PCEF support

- Bugfix: cast ignore ras ids to int

- Bugfix: read min/max event time directly from defs

- Bugfix: fix group usages of connection analysis

- Feature: Add User ID Attribute to Search User Page

- BugFix: Threshold Bytes/Seconds did not trigger internal reauthorize due to bad merge

- BugFix: Threshold calculations was went south due to bad merge

- BugFix: Second username change fail in case of internal relogin

- Improvement: get method for SNMP client

- Bugfix: fix internal relogin answer to main proc

- Bugfix: Race condition might cause internal relogin to fail

- Bugfix: Autorenew would fail if user has next group and current group does not have group_credit

- Bugfix: fix limit port binding

- Bugfix: fix periodic acc failed login

- Bugfix: fix non-working periodic accounting plugin

- Bugfix: Some mikrotik devices send mschapv2 flags as non-zero which is not compliant by standard but leads to wrong passwords

- Feature: optionally do autoRenew on abs expired user

- Bugfix: log deposit_type in deposit change

- Cleanup: clean md5 response

C_lan_diameter Branch

تگ 401 تا 402 در تاریخ ۱۳۹۶/۰۷/۲۶

Bugfix: #1685 snifflogger not working 

our database connection did not load since python uses double underscore
for private method hence we changed the double underscore to single
so it could be intialized and return connection for snifflogger database.

BugFix: Previous usage for maximum rule traffic usage with a custom field used in multiple rules were not calculated correctly

BugFix: applicableUntil of max rule traffic usage plugin with same custom field as value of multiple rules were not working correctly, leading to incorrect usage calculations

Improvement: Updated sample configurations

BugFix: Outgoing radius packets initiated from ws processes were not logged

Bugfix: Fix a Change in Previous Commit

Bugfix: #1682 Remove `addToBlackList` Cause We Don't Use It Any More 

This method belonged to `UserPool` class which was removed in
master/slave feature.

Bugfix: #1486 Add SYNC_DATA Config Header

Bugfix: #1486 Change SYNC_DATA Header Name to SYNC_DNS

Improvement: Allow passing extra arguments to canStay

Improvement: Send ras msg in internetReauthorize so charge plugin can use reauthorize_use_bytes in calculations

Improvement: Redirect user to user info after credit change using GET instead of Post, so user info page will be refreshable

Bugfix: #1663 Removing Leaves Should be Ran in Online Worker

Improvement: Make less query while loading user to optimize performance

Feature: #1645 add a white list for referrers to config 

add a white list to section "INTERFACE" inside IBSng.conf.default

Bugfix: #1486 Add Config Plus New Option For AD DNS Sync

Improvement: #1550 Change Date Limit from 10 to 15 Years

Improvement: Allow abs exp date up to 15 years

BugFix: Empty cisco_isg_failed_service was not supported with FAILED_USER_L4 in cisco_isg_failed_method

BugFix: Force Auto Recharge on zero credit was not working if deposit was added while user is offline

BugFix: Max Rule Time Usage previous value were not shown in edit template

BugFix: A service rule without negative credit while user has credit limit less than zero would not trigger a reauthorize

NewFeature: auto_recharge_force_on_zero_credit Feature to perform auto recharge even when credit limit is fewer than zero and negative credit has been set

BugFix: Radius Queue Statistics were not working on multiple machines

Improvement: FAILED_USER_L4 will always send services cisco_isg_failed_service separated by comma and appended by __Unlimited

BugFix: Radius packets were missed because of typo

BugFix: Partial log retry on lock were incorrect causing massive retry of pointless partial logs

BugFix: Normal stop were not working because of typo

BugFix: Make sure we have enough user pool capacity and we will not leak loaded user objects

BugFix: Non-priority queue is not multi login friendly causes locks being hold while user is in queue

Improvement: Allocate db pools in 3 batches to prevent everyone allocate a handle on start situation

BugFix: Online locks should always be distributed as user has no native process

BugFix: Server name should not be in online_users table anymore

BugFix: Worker Queue was malfunctioning leading to incorrect counter values

BugFix: Various new mm fixes

BugFix: Internal Relogin were not working

Improvement: Throttle getting distributed events if worker queue are high

BugFix: Blocking mode of radius job polling caused reply loop not to work. Fixed by running another thread for receive of replies.

Improvement: Use ippools directly in RADIUS_WORKER processes to prevent expensive mainEvaluate calls

BugFix: Throttled job polling loop did not wait for job queues to become lower than threshold and just waited once

BugFix: Adding periodic accounting while user is online would cause user info page not to work

Improvement: Reversed the job acquire mechasim from radius workers pushing to online workers into online workers pull from radius workers

BugFix: partial log event time was incorrect

Improvement: Change user pool size based on process. Online workers does not need to keep loaded users anymore

BugFix: Idle users were not compatible with new multi master

Enhancement: Remove Recalc In Queue Detection as it's not applicable anymore

BugFix: Can Stay Runner were not compatible with new multi master

NewFeature: Redis Backed, Shared State IPPools.

BugFix: Clear user were not working with new multi master

BugFix: Internal Relogin were not working with new multi master

NewFeature: Distributed Event and Delayed Queue Feature

First Working version of stateless online workers


تگ 400 تا 401 در تاریخ ۱۳۹۶/۰۷/۱۵

Bugfix: Fix Sanitization to Ignore Some Strings 

By applying sanitization on web request, following to character are
recognized as html tags and so will be ignored (deleted) from request.

Improvement: Do not kill service instances as change service will take care of that

NewFeature: If Not Sub Service Charge Rule Attribute to activate a service only when another service is not activated

Feature :#1647 add step for ippool percentage in config 

step number added for ippool percentage in config file in INTERFACE
section with interface_ippool_step TAG

تگ 397 تا 400 در تاریخ ۱۳۹۶/۰۷/۰۳

Support Request: #1555 Add User Import Log

Feature: Port auth_event Plugin on C_diameter Branch

Feature: #1460 Send User Events on Message Broker

Bugfix: Include Missing `bm_id` in `applyBwLimit` Method

Improvement: Stop Clearing Terminal on IBSng Startup

Bugfix: #1634 fix connection log report problem 

this problem caused because of new implementation for xss prevention
fixed the place of the code.

Improvement: #1617 Add PubSub to Invoice Profile

Feature: #1553 Add PubSub to Custom Field 

This commit is copy/paste of PR No. 23 which has been done by Morteza
Nourelahi Alamdari on development branch. Because this is a priority
for us in stable branch we create this commit.

Bugfix: #1525 bulk action not found error fix 

it isn't fixed completely yet since ws_workers are more than 1, and
bulk action instance may not have action_id in every ws_worker hence
we face this error. so we sated action_id to $_SESSION so we will
keep it in front side for getting info about it.

Improvement: Add `requests` Module to Python Dependency Packages

Improvement: #1618 Correct Space between Codes

Feature: #1239 Add `online_payment.getAvailableGatewayTypes` Method 

This method have been copy/paste from staging branch because it's been
needed for Wispot project and we can't wait for merging staging branch
to stable one.

Feature: #1611 Add Forgot Password to User Web Panel 

Actually this feature has been implemented before in staging branch by
Saeed Rasooli. Since we are not ready now to merge our staging branch
to stable one, so we have some how copy/paste the changes (with some
cleanup) from staging branch to answer customer request.

Improvement: #1618 Return admin_id in addNewAdmin Method

Bugfix: Fix NTLM Auth Incorrect Variable

Feature: #1572 fix xss security problem and sql injection 

xss csrf and auto complete in form have been fixed .

Bugfix: #1589 TEMPORARY Bug Fix on LDAP Sync Action

Improvement: #1589 Change LDAP Delete/Insert to Update/Insert Action 

Set timeout from "2" to "5" in ldap connection.

Improvement: #1486 Update Bash File Name

Improvement: #1486 Update "return_exception" Docstring

Improvement: #1486 Implement AD Logon Host Feature

Cleanup: UrlDumpDBPool Cleanup

Bugfix: #1515 fix all online graph's time line in user and admin panel 

all the time line issues in both admin panel and user panel for real time graphs have been fixed for branches without mongoDB.

Improvement: #1550 Validate Absolute Expiration Date Input 

Check the epoch time that it won't pass 10 years from now.

Bugfix: #1501 fix group_id and username showing problem in snifflogger

Improvement: Online state keeper refactor and code cleanups to improve performance and correctiveness

Improvement: Ensure we don't leak redis connections in rare cases of pipelines

Cleanup: 0 does not need to be passed to price modifier

Improvement: Made online worker concurrency configurable which is count of worker threads in each online worker. The actual concurrency would be twice this number because threads sending requests to user workers and threads receiving results from user workers are separated

Improvement: Change the order which db handles are used to enable idle checking. It was a FIFO queue and now its LIFO.

Feature: #1533 Exclude parameters from LDAP info in Web Panel 

now we can set the ldap options to show in setting -> advanced -> advanced config -> LDAP PARAM EXCLUDE

NewFeature: Rebin Integration Support

Feature: #1490 Compatible "init_bw_manager" with All Kernel Versions

تگ 397 در تاریخ ۱۳۹۶/۰۶/۰۵

This branch is the merge of C_lan_acc into C_diameter branch.


C_diameter_mongo Branch

تگ 401 تا 402 در تاریخ ۱۳۹۶/۰۷/۲۶

Bugfix: #1694 1694 Duration Reporting in Connection log Raise Error

BugFix: Previous usage for maximum rule traffic usage with a custom field used in multiple rules were not calculated correctly

Bugfix: #1384 Add dbname In Mongo Connection URL

Improvement: Add try..except in Save BW Usage Method

BugFix: applicableUntil of max rule traffic usage plugin with same custom field as value of multiple rules were not working correctly, leading to incorrect usage calculations

Improvement: Updated sample configurations

BugFix: Outgoing radius packets initiated from ws processes were not logged

Bugfix: Fix a Change in Previous Commit

Bugfix: #1682 Remove `addToBlackList` Cause We Don't Use It Any More 

This method belonged to `UserPool` class which was removed in
master/slave feature.

Improvement: Allow passing extra arguments to canStay

Improvement: Send ras msg in internetReauthorize so charge plugin can use reauthorize_use_bytes in calculations

Improvement: Redirect user to user info after credit change using GET instead of Post, so user info page will be refreshable

Bugfix: #1663 Removing Leaves Should be Ran in Online Worker

Improvement: Make less query while loading user to optimize performance

Improvement: Make less query while loading user to optimize performance

Improvement: Lowered threshold for throttling online worker task loop. It was too high causing high load online worker to send out many late answers

Feature: #1645 add a white list for referrers to config 

add a white list to section "INTERFACE" inside IBSng.conf.default

Improvement: #1550 Change Date Limit from 10 to 15 Years

Improvement: Allow abs exp date up to 15 years

BugFix: Empty cisco_isg_failed_service was not supported with FAILED_USER_L4 in cisco_isg_failed_method

BugFix: Force Auto Recharge on zero credit was not working if deposit was added while user is offline

BugFix: Max Rule Time Usage previous value were not shown in edit template

BugFix: A service rule without negative credit while user has credit limit less than zero would not trigger a reauthorize

NewFeature: auto_recharge_force_on_zero_credit Feature to perform auto recharge even when credit limit is fewer than zero and negative credit has been set

BugFix: Radius Queue Statistics were not working on multiple machines

BugFix: Radius Queue Statistics were not working on multiple machines

Improvement: FAILED_USER_L4 will always send services cisco_isg_failed_service separated by comma and appended by __Unlimited

BugFix: Radius packets were missed because of typo

BugFix: Partial log retry on lock were incorrect causing massive retry of pointless partial logs

BugFix: Normal stop were not working because of typo

BugFix: Make sure we have enough user pool capacity and we will not leak loaded user objects

BugFix: Non-priority queue is not multi login friendly causes locks being hold while user is in queue

Improvement: Allocate db pools in 3 batches to prevent everyone allocate a handle on start situation

BugFix: Online locks should always be distributed as user has no native process

BugFix: Server name should not be in online_users table anymore

BugFix: Worker Queue was malfunctioning leading to incorrect counter values

BugFix: Various new mm fixes

BugFix: Internal Relogin were not working

Improvement: Throttle getting distributed events if worker queue are high

BugFix: Blocking mode of radius job polling caused reply loop not to work. Fixed by running another thread for receive of replies.

Improvement: Use ippools directly in RADIUS_WORKER processes to prevent expensive mainEvaluate calls

BugFix: Throttled job polling loop did not wait for job queues to become lower than threshold and just waited once

BugFix: Adding periodic accounting while user is online would cause user info page not to work

Improvement: Reversed the job acquire mechasim from radius workers pushing to online workers into online workers pull from radius workers

BugFix: partial log event time was incorrect

Improvement: Change user pool size based on process. Online workers does not need to keep loaded users anymore

BugFix: Idle users were not compatible with new multi master

Enhancement: Remove Recalc In Queue Detection as it's not applicable anymore

BugFix: Can Stay Runner were not compatible with new multi master

NewFeature: Redis Backed, Shared State IPPools.

BugFix: Clear user were not working with new multi master

BugFix: Internal Relogin were not working with new multi master

NewFeature: Distributed Event and Delayed Queue Feature

First Working version of stateless online workers


تگ 400 تا 401 در تاریخ ۱۳۹۶/۰۷/۱۵

Bugfix: Fix Sanitization to Ignore Some Strings 

By applying sanitization on web request, following to character are
recognized as html tags and so will be ignored (deleted) from request.

Improvement: Do not kill service instances as change service will take care of that

NewFeature: If Not Sub Service Charge Rule Attribute to activate a service only when another service is not activated

Add Group Attr Checker for BW Save Flag

Bugfix :#1639 fix user bw_graph time in mongo branch for user panel

Feature :#1647 add step for ippool percentage in config 

step number added for ippool percentage in config file in INTERFACE
section with interface_ippool_step TAG

تگ 398 تا 400 در تاریخ ۱۳۹۶/۰۷/۰۳

Support Request: #1555 Add User Import Log

Feature: Port auth_event Plugin on C_diameter Branch

Feature: #1460 Send User Events on Message Broker

Improvement: Stop Clearing Terminal on IBSng Startup

Bugfix: #1613 Connection Log Report Problem

Bugfix: #1634 fix connection log report problem 

this problem caused because of new implementation for xss prevention
fixed the place of the code.

Improvement: #1617 Add PubSub to Invoice Profile

Feature: #1553 Add PubSub to Custom Field 

This commit is copy/paste of PR No. 23 which has been done by Morteza
Nourelahi Alamdari on development branch. Because this is a priority
for us in stable branch we create this commit.

Bugfix: #1525 bulk action not found error fix 

it isn't fixed completely yet since ws_workers are more than 1, and
bulk action instance may not have action_id in every ws_worker hence
we face this error. so we sated action_id to $_SESSION so we will
keep it in front side for getting info about it.

Improvement: #1618 Correct Space between Codes

Feature: #1239 Add `online_payment.getAvailableGatewayTypes` Method 

This method have been copy/paste from staging branch because it's been
needed for Wispot project and we can't wait for merging staging branch
to stable one.

Feature: #1611 Add Forgot Password to User Web Panel 

Actually this feature has been implemented before in staging branch by
Saeed Rasooli. Since we are not ready now to merge our staging branch
to stable one, so we have some how copy/paste the changes (with some
cleanup) from staging branch to answer customer request.

Improvement: #1618 Return admin_id in addNewAdmin Method

Feature: #1572 fix xss security problem and sql injection 

xss csrf and auto complete in form have been fixed .

Add Save BW Usage Condition For Specified Users Only

Bugfix: #1515 fix all online graph's time line in user and admin panel 

all the time line issues in both admin panel and user panel for real time graphs have been fixed for branches without mongoDB.

Improvement: #1550 Validate Absolute Expiration Date Input 

Check the epoch time that it won't pass 10 years from now.

Fix: Convert getTopConnectionCount Query to MongoDB

Feature: #1146 Add New Permission "CHANGE USER LIMIT MAC" 

From now on, only admins with specified permission can modify the
limit mac attribute of users. This permission has "CHANGE USER
ATTRIBUTES" permission as it's parent.

Fix: unique_id And unique_id_value Position 

We needed to move unique_id and unique_id_value's position to details
to have it in Report/Connection Logs

Bugfix: Fix 'before_credit' KeyError in PSQL to Mongo Conversion

تگ 394 تا 398 در تاریخ ۱۳۹۶/۰۶/۰۸

Improvement: Online state keeper refactor and code cleanups to improve performance and correctiveness

Improvement: Ensure we don't leak redis connections in rare cases of pipelines

Cleanup: 0 does not need to be passed to price modifier

Improvement: Made online worker concurrency configurable which is count of worker threads in each online worker. The actual concurrency would be twice this number because threads sending requests to user workers and threads receiving results from user workers are separated

Improvement: Change the order which db handles are used to enable idle checking. It was a FIFO queue and now its LIFO.

NewFeature: Rebin Integration Support

Improvement: Implemented fast shutdown specifically to flush online user count queue before force killing.

Cleanup: Cleanup mpi adaptor codes

Enhancement: Refactor dbpool to use less connections to db. Also it releases db handles if they are not used more than defs.DB_POOL_IDLE_RELEASE_TIME which is 15 minutes by default

BugFix: Not flushing charge rule cache on reload might cause problems with state changes

BugFix: Fixed double caching loaded user in user_obj causes loaded user not to be reloaded correctly

BugFix: Multi Master and LIMIT LOGIN ADDR permissions were not compatible

BugFix: Proxy of requests with auth_pass did not work

Improvement: Missing instance during killAllInstances operation is normal and should not be logged into error log

BugFix: Negative CPMB caused credit notification to go into a very large loop

BugFix: Fixed typo in error message

Improvement: Kill and Clear must be run on owner server

BugFix: Fast Start was not working with license

BugFix: Fast Start was not working with license

BugFix: Fix kill was not working due to change in ras redis attributes

BugFix: We should raise Exception if we can not move a user

BugFix: We should raise Exception if we can not move a user

BugFix: Background and font color of notification was the same making them unreadable

BugFix: Template closing tag was incorrect

BugFix: Background and font color of notification was the same making them unreadable

BugFix: Template closing tag was incorrect

Improvement: Pass user_obj to user_workers through queues instead of redis to improve performance

NewFeature: Ability to run on multiple machines and fast restart

Bugfix: #1520 Fix Connection Log Access in User Panel 

When a request calls the "report.getConnection" method, requester will
be a Normal User which don't have "hasPermOrGOD" method. This have
been handled in this commit.

BugFix: Enhancements in online object to prevent user object corruption

Improvement: Run two threads for handling each user worker instead of one to improve performance

Improvement: shutdown users enhancement on clear only scenario

BugFix: Fix Extra charge was applying with a large delay due to slow mid night events

BugFix: Daily event start debug log were never written

Improvement: Refactor successful auth boolean

BugFix: Refactor multilogin plugin to not keep multilogin_not_allowed in plugin state and reduce memory usage

BugFix: Fixed typo causing user worker feeder loop to slow down

Improvement: Improve thread latency

BugFix: Some redis dbs were not flushed on startup

Improvement: Do not use multiple dbs for ras data. Redis people discourage using multiple dbs

Improvement: Process requests of RADIUS_WORKER in 4 threads instead of 1 to improve performance

Improvement: Add autosync_on_change_service option to pcef rases to resync instead of change service This fixes douran bug which does not understand change services

Improvement: Add auth by caller id support to huawei ras

تگ 385 تا 394 در تاریخ ۱۳۹۶/۰۵/۲۳

Improvement: Allow defining two other redis instances for radius and online_user_storage for redis clustering

BugFix: 3GPP location info is in binary format and should be converted

Feature: Add "FULL ACCESS CONNECTION LOGS" Permission 

 Admins with this permission can set all filters of connection
 log/usage report (like a GOD admin). This permission has "SEE
 CONNECTION LOGS" dependency.

Bugfix: Fix Result Per Page Limit of Connection Log 

 This problem occured after limiting users on accessing certain filters
 in connection log report page, where setting result per page parameter
 has no effect on output result.

Feature: #1477 fix access for non GOD users to connection usage

Feature: #1477 add control permission for non GOD users to connection usage 

 remove advanced and group tabs and remove isp from conditions tab.
 and the maximum range of login_from , login_to is 1 month now.

BugFix: runDebugCode were not working in WS_WORKERS

BugFix: extractRoutes paniced in case ras set routes in prefix bit

NewFeature: Allow having multiple webservice processes

BugFix: Typo in expirePIQuery resulted in no PI expiration

BugFix: Session ID were not shown in connection log report

Bugfix: #1412 Change Back Using Event Time Instead of Remaining Time 

 In `canStayOnlineResult` when we intend to use `newRemainingTime` we
 have to make sure about exact value of this parameter (no estimation
 here), otherwise we have to use `newNextEventTime`.

Bugfix: #1412 Using `get` for Fetching Threshold Values

Bugfix: #1412 Fix Using of Thresholds in Can Stay Online Result 

 If can stay online result object contains `remaining_bytes` or
 `remaining_time` we will fill our `session_dict` with
 `threshold_bytes` and `threshold_secs` respectively.
   
 Now if an alive packet arrives and one of these thresholds meet, we
 run `canStayOnline` methods of user plugins (by force) to recalculate
 the parameters. This will fix Periodic Accounting and Notification
 Profile plugins functionality.
   
 Note: We have error threshold of twice the interval between alive
 packets, meaning, we may run `canStayOnline` methods late by specified
 time.

Improvement: Increase udp memory buffers to improve radius server performance

Improvement: Use blocking recvfrom instead of select as now we have multiple processes recvfrom same socket

Bugfix: Add `routes` Parameter to Cisco PCEF `provisionRequest` Method

NewFeature: Implemented pretty start using blessings to emphasize important events on startup

BugFix: Diameter process should not process any radius packets

BugFix: Diameter process should not process any radius packets

BugFix: Fixed duplicate packets were not working

Improvement: Moved RADIUS_AUTH and RADIUS_ACCT into RADIUS_WORKER for better scalability

BugFix: Fixed douran provisioning did not support routes

BugFix: Framed-Route passing to pcef were not working on huawei rases

BugFix: Fixed typo which caused Traffic Periodic Accounting without cpmb to not work

BugFix: AppliedChargeRule.getChargeRule should return None if charge rule is no longer available

Bugfix: Add Two Attributes to Radius Dict; `byte` and `ipv6addr`

Improvement: Some Changes for Non GOD Report of Connection Log 

 Also Add new attribute types of 3GPP.

Feature: #1447 add permissions for non God admin in connection log 

 added some permission for non god admins witch non god admins wont see some tabs in connection log page and range of login time will be at most  one month .

BugFix: CPMB input validation was incorrect

Improvement: API change bw were not working in C_diameter based branches

NewFeature: 3GPP attribute support in huawei ras

BugFix: Fixed typo in sendCOA caused log not to be written in console for COAs

BugFix: Append a * to custom field attribute names to indicate its a custom field and prevent clash of request names with conditions

Improvement: Allow setting negative cpmb as a mean for services to reduce from master traffic

Bugfix: #1455 Change Priority of User Rad Attr Over Charge Rad Attr 

 In some cases (such as "Filter-Id" attribute) Rases will only consider
 the latest attribute in case two (or more) attributes have same
 keys. Because user radius attributes have priority over charge radius
 attribute by setting it's priority to 7, it will run after charge plugin
 (which it's priority is 6) so the considered attribute value will be
 the user's one.

Feature: #1397 Remove Admin Condition From User Audit Log 

 We replaced Admin condition in User Audit Log page with User ISP
 condition where the records will be filtered based on accessable
 user(s) in ISP instead of accessable Admin(s) of ISP.

NewFeature: Support for routes in pcef user awareness

Feature: #1462 add failed online reason to user information page 

 if there was a reason for failed online it will be shown on user information page .

Improvement: Enhanced traffic periodic accounting to use can stay remaining bytes and internal reauthorize

BugFix: failed_login_assign_ip was not applied to routeip and netmask plugins

BugFix: Update python dependency file

BugFix: Caller ID were not available in mongo connection log

BugFix: IPHost-Addr attribute were changed in new huawei dictionary

BugFix: An error during periodic credit change checkAll might cause skipping of change credit for some users

Improvement: Script to reload radius dictionary. Should be run in all radius workers

BugFix: Huawei DNS attributes were changed in new dictionary

BugFix: Duplicate Mac/Port did not work after last change

Improvement: A handy script to kill all failed users

BugFix: Failed users does not need internal relogin as killIfFailed would take care of that

BugFix: getOnlineUsersIterItems was returning user_id as string instead of integer

BugFix: Fix typo in table name

Improvement: New client script to clear all acct not started users

BugFix: Huawei ras was not compatible with new dictionary file

Bugfix :#1441 real time online graphs don't show 

 this problem caused cause of bad convert formats . config edit added.

Improvement: Updated huawei dictionary

Bugfix :#1441 real time online graphs don't show 

 this problem caused cause of bad convert formats . exception handling added.

Bugfix :#1441 real time online graphs don't show 

 this problem caused cause of bad convert formats .

Bugfix: #1417 Minor Cleanup

Bugfix: #1417 Fix Load Balancing Functionality of IPPool Plugin 

 The `ippool_main.getLoader().getIPpoolByID` will return either an
 "IPPool" or "LoadBalancingIPPool" object. In later case the object has
 no `getContainer` method. Instead we use `setIPInPacket` method which
 is available in both objects and has the same functionality.

Cleanup: Removed ONLINE_CHECK_MULTI_LOGIN from advanced config as it was duplicate functionality with multi login plugin

BugFix: Failed logins should not be counted into multi login. Now allow up to 3 failed logins in addition to normal master instances.

Feature: #1317 Add ISP Filter to `getListOfCharges` Charge Action

Feature: #1317 Add ISP Filter to `getListOfGroups` Group Action

BugFix: IP Pool plugin was raising None if ip is not available in ippool anymore

Improvement: Can Stay Result now have 4 variables 

 Remaining Time: Charging and Authorization remaining Time
 Next Event Time: Used by plugins need to set a next event
 Improtatnt Event Time: This overrides MIN_USER_EVENT_TIME
 Remaining Bytes: Bytes remaining until next event

Improvement: Internet Reauthorizes which triggered auto recharge were rejected incorrectly

BugFix: Charge.getRuleWithID should return None if rule id does not exists

NewFeature: diameter_prevent_cctime_avalanche attribute in diameter ras controls if cc-time should be distributed across half of a update accounting time

NewFeature: Persistent charge rule cache to use a LFU cache for keeping applied charge rule data

BugFix: Diameter terminate would reset session dict by mistake

BugFix: Diameter response time can be higher than usual radius authentication

BugFix: Packet storage should not be opened in online worker due to performance penalty

NewFeature: Added support to specify unix socket or ip address of syslog through syslog_destination in config file

Bugfix: Fix Minor Bug

BugFix: Kill failed user if deposit has been changed in hope of auto renew or auto recharge

Improvement: added REQUIRE_INTERNAL_RELOGIN flag to AttributeHandler to control if internal relogin is required for updating the specific attribute. This will reduce number of internal relogins required

BugFix: IBSErrors are mostly login related and should not be logged in error.log

BugFix: Showing session id in mongo logs were not working

BugFix: Failed and Idle login automatic detection on re-online were not working

Cleanup: Removed unused DEBUG_ERRORS, DEBUG_WARNINGS, DEBUG_THREADS from defs and logging

Cleanup: Removed duplicate defs entries

Improvement: Allow changing including of seconds in date through advanced config

Revert "Bugfix :#1424 real time bw graph timeline problem" 

 This reverts commit ffc1ef6a7921c6aa205a50bb6d22a93318d5581b.
  
 Since graphs in Mongo branch work fine, we don't need this commit.

BugFix: _handlePacketResponse signature was changed while some usages were missing

Bugfix :#1436 change QOS defs propertis to accept string input

BugFix: Fix small memory and thread leak problem in thread wrappers

BugFix: Douran ras was using charging name to be passed as rating group. It should be extracted from sub service name

NewFeature: Reject cache configuration has been moved to ras attributes. [REJECT CACHE] Configuration section is now deprecated.

NewFeature: Diameter translated packets are now written into DIAMETER/diameter_translation.log instead of filling up RADIUS/auth.log

Improvement: Diameter packets will use new packet codes DiameterCCRInit,DiameterCCRUpdate,DiameterCCRTerm,DiameterCCASuccess,DiameterCCAFailure instead of radius ones

BugFix: True or False should not be used in defs variables. 1 or 0 should be used instead

Bugfix: Replace `__bool__` With `__nonzero__` 

 `__bool__` data model is replacement of `__nonzero__` in python 3.x;
 since we're still on python 2.x we have to still use `__nonzero__`

NewFeature: Support for setting cisco_isg_failed_method attribute of cisco isg ppp ras to FAILED_USER_L4 which causes failed user to have normal ip assignments

Bugfix :#1424 real time bw graph timeline problem 

 time line problem solved using utc time guide.

Improvement: don't log wrong password

Improvement: Show session id (unique_id_value in fact) in connection log report

Improvement: Show seconds by default in all reports

BugFix: if pkt was checking attributes length while it should be always true

Improvement: Log storage address in radius logs for debugging purpose

Improvement: Checkonline was running too fast causing Late Packets during it's run

Improvement: Better logging for packet storage

BugFix: Prevent double slashes for shared storage file paths

BugFix: threadpool terminate and close methods were not working

Improvement: Log ibs request id into radius log to make debugging easier

BugFix: In some situations, cisco pcef session did not stop after master session has been disconnected. Make sure the session is cleaned up by first sending pod if session pcef dict is not available and second optionally send a pod on each start packet

BugFix: Fixed a race condition that might cause packet storage files not to be deleted

Bugfix: #1367 search_expired_user.php check_all_users problem 

 multi select problem for users has been handled.

BugFix: Policy worker server log were not filtered

BugFix: snmp agent were not working due to xmlrpc limitations. Converted client to use jsonrpc instead

BugFix: snmp agent were not working due to xmlrpc limitations. Converted client to use jsonrpc instead

BugFix: Policy worker server log were not filtered

Cleanup: Refactor threadpool to be more efficient and simple. Also there would be no need to tune number of threads for different sizes of system as threadpool would grow automatically

Improvement: Do not log unnecessary attributes in connection log

Cleanup: Do not log soap handler registeration if debugging is off

Cleanup: Removed not migrated ras folder

BugFix: Exception Logs were written twice

BugFix: IP Conflict happens during re-online because conflicted session did not get kicked out

Improvement: Include seconds in date representation

Improvement: Timedelta representation for kill time is ugly. Use default date format instead

BugFix: Exception Logs were written twice

BugFix: IP Conflict happens during re-online because conflicted session did not get kicked out

Improvement: Include seconds in date representation

Improvement: Timedelta representation for kill time is ugly. Use default date format instead

تگ 376 تا 385 در تاریخ ۱۳۹۶/۰۴/۱۵

- Bugfix: #1408 Port call_id_blacklist plugin to C branch

- BugFix: Moved SOAP HTTP Server from builtin python library into a local library and swap select.select to poll.epoll

- BugFix: Stop priority should be higher

- BugFix: Fixed internal relogin might raise (harmless) error if called multiple times

- BugFix: Cisco VPDN was nagging incorrectly if user does not have in ras bw

- BugFix: Not finding session dict during session start is not an error and should not be reported as an error

- BugFix: Maximum reauthorize quotas did not apply if effective charge rule does not have CPMB

- BugFix: Cisco PCEF stop awareness should remove session from activated sessions

- BugFix: pass_remaining_volume and pass_remaining_duration ras attributes were not applied in cisco isg races

- BugFix: Internal Relogin would cause ip conflict if ippool is defined in user or charge level

- BugFix: Internal Relogin kill reason was lost due to not saving user state

- BugFix: Send service resync requests after internal_relogin with a delay to make sure all instances has been relogged in before resync

- Improvement: Clean up imports so unnecessary components would not be initialized in listener processes

- BugFix: User Worker Queue did not run requests in order which their arrived even if they had equivalent priority

- BugFix: ID generation was not working correctly for multiple processes. Fixed by separating port range of each radius worker

- Improvement: Cache time series dynamic register statistics to decrease redis hgetall calls

- BugFix: Python generators are not threadsafe causes some packet might not get a response

- BugFix: Fixed Acct-Session-Time was incorrectly interpreted for diameter ras

- Fixed missing diameter session would cause a recursive loop until next packet arrives

- BugFix: Fixed a race condition which might happen because isp online is no longer available

- BugFix: appendleft should not return the right most item. This is not the behaviour of python deque and perform an extra loads for each append

- BugFix: Radius runs out of threads for deployments with large amount of processes

- BugFix: Drop re-online request if user is already online

- NewFeature: Show Policy Worker Statistics in summary view page

- BugFix: TimeSeries was returning error if count of time based stat was zero

- NewFeature: Log Server Initiated Radius requests under /var/log/IBSng/RADIUS/server_initiated.log

- Improvement: Make onlines tables UNLOGGED to improve performance

- BugFix: Fix RedisDict might get into infinite loop during iterations

- BugFix: Relative expiration date was setting incorrect next event during canStay

- NewFeature: Implemented Rate limiter for ras kill, change service, coa and pcef provision server initiated requests

- BugFix: Defs change was not propagated through other processes

- BugFix: getAllStatsByStatCategory was not threadsafe

- BugFix: RedisDict iterators were not behaving correctly

- BugFix: Converted admin request limit to rratelimit library. The feature were not working before

- Improvement: Change print format to stand out between logs

- BugFix: Called-Station-Id was passing as service causing problem with charge rule matching

- Improvement: Sort rates dashboard widget values

- BugFix: Terminate Cause and Login Error rates were not working

- BugFix: Prevent unnecessary error logs

- Fix: Show Exact Value for ISP's on Online Users

- Feature : #1361 change the sub service qos name to limited drop down 

 change  the sub service qos name to limited drop down options .

- BugFix: empty or 0.0.0.0 as xmlrpc bind ip would cause proxied request not to work

- Improvement: Logging support for snmp passpersist agent

- BugFix: Logrotate did not include POLICY_WORKER and DIAMETER logs so rotation did not happen for them leading to log folder getting full

- BugFix: Remove policy request queue generator as it was not thread safe leading to messed up state of queues

- Feature : #1360 edit charge priority from 1-10 to 1-50 

 edit charge priority to 1-50 and it's backend error and validation rules.

- BugFix: ignore_cpmb were not working on cpmb_download and cpmb_upload

- Improvement: Re-order if conditions to prevent unnecessary redis lookup

- BugFix: CheckOnline was too slow

- Improvement: Enable SUB_SERVICE_CHECK By default and disable syncing during re-online which puts high load on BRAS when IBSng service is restarted

- BugFix: Kill User if a COA activation failed on cisco ISG ras

- BugFix: Import library fixes in more locations to make sure it will be loaded in all processes before other libraries

- BugFix: Kairos might be initialized incorrectly because system redis library was imported instead of local one

- BugFix: no need to pass ip:addr-pool in cisco-avpair for in ras ippool to work. Fixes incompatibility with asr9k


تگ 375 تا 376 در تاریخ ۱۳۹۶/۰۳/۲۷

- Bugfix: #1339 Load Invoice Main in Online Worker

- Bugfix : #1300 delete realtime request log interface 

 real time request log  deleted  from interface because the source code of this link  had been deleted before .

- Bugfix: #1339 Load Invoice Main in Online Worker

C_lan_diameter_mongo Branch

تگ 401 تا 402 در تاریخ ۱۳۹۶/۰۷/۲۶

Bugfix: #1694 1694 Duration Reporting in Connection log Raise Error

Bugfix: #1685 snifflogger not working 

our database connection did not load since python uses double underscore
for private method hence we changed the double underscore to single
so it could be intialized and return connection for snifflogger database.

BugFix: Previous usage for maximum rule traffic usage with a custom field used in multiple rules were not calculated correctly

Bugfix: #1384 Add dbname In Mongo Connection URL

Improvement: Add try..except in Save BW Usage Method

BugFix: applicableUntil of max rule traffic usage plugin with same custom field as value of multiple rules were not working correctly, leading to incorrect usage calculations

Improvement: Updated sample configurations

BugFix: Outgoing radius packets initiated from ws processes were not logged

Bugfix: Fix a Change in Previous Commit

Bugfix: #1682 Remove `addToBlackList` Cause We Don't Use It Any More 

This method belonged to `UserPool` class which was removed in
master/slave feature.

Bugfix: #1486 Add SYNC_DATA Config Header

Bugfix: #1486 Change SYNC_DATA Header Name to SYNC_DNS

Improvement: Allow passing extra arguments to canStay

Improvement: Send ras msg in internetReauthorize so charge plugin can use reauthorize_use_bytes in calculations

Improvement: Redirect user to user info after credit change using GET instead of Post, so user info page will be refreshable

Bugfix: #1663 Removing Leaves Should be Ran in Online Worker

Improvement: Make less query while loading user to optimize performance

Improvement: Make less query while loading user to optimize performance

Improvement: Lowered threshold for throttling online worker task loop. It was too high causing high load online worker to send out many late answers

Feature: #1645 add a white list for referrers to config 

add a white list to section "INTERFACE" inside IBSng.conf.default

Bugfix: #1486 Add Config Plus New Option For AD DNS Sync

Improvement: #1550 Change Date Limit from 10 to 15 Years

Improvement: Allow abs exp date up to 15 years

BugFix: Empty cisco_isg_failed_service was not supported with FAILED_USER_L4 in cisco_isg_failed_method

BugFix: Force Auto Recharge on zero credit was not working if deposit was added while user is offline

BugFix: Max Rule Time Usage previous value were not shown in edit template

BugFix: A service rule without negative credit while user has credit limit less than zero would not trigger a reauthorize

NewFeature: auto_recharge_force_on_zero_credit Feature to perform auto recharge even when credit limit is fewer than zero and negative credit has been set

BugFix: Radius Queue Statistics were not working on multiple machines

BugFix: Radius Queue Statistics were not working on multiple machines

Improvement: FAILED_USER_L4 will always send services cisco_isg_failed_service separated by comma and appended by __Unlimited

BugFix: Radius packets were missed because of typo

BugFix: Partial log retry on lock were incorrect causing massive retry of pointless partial logs

BugFix: Normal stop were not working because of typo

BugFix: Make sure we have enough user pool capacity and we will not leak loaded user objects

BugFix: Non-priority queue is not multi login friendly causes locks being hold while user is in queue

Improvement: Allocate db pools in 3 batches to prevent everyone allocate a handle on start situation

BugFix: Online locks should always be distributed as user has no native process

BugFix: Server name should not be in online_users table anymore

BugFix: Worker Queue was malfunctioning leading to incorrect counter values

BugFix: Various new mm fixes

BugFix: Internal Relogin were not working

Improvement: Throttle getting distributed events if worker queue are high

BugFix: Blocking mode of radius job polling caused reply loop not to work. Fixed by running another thread for receive of replies.

Improvement: Use ippools directly in RADIUS_WORKER processes to prevent expensive mainEvaluate calls

BugFix: Throttled job polling loop did not wait for job queues to become lower than threshold and just waited once

BugFix: Adding periodic accounting while user is online would cause user info page not to work

Improvement: Reversed the job acquire mechasim from radius workers pushing to online workers into online workers pull from radius workers

BugFix: partial log event time was incorrect

Improvement: Change user pool size based on process. Online workers does not need to keep loaded users anymore

BugFix: Idle users were not compatible with new multi master

Enhancement: Remove Recalc In Queue Detection as it's not applicable anymore

BugFix: Can Stay Runner were not compatible with new multi master

NewFeature: Redis Backed, Shared State IPPools.

BugFix: Clear user were not working with new multi master

BugFix: Internal Relogin were not working with new multi master

NewFeature: Distributed Event and Delayed Queue Feature

First Working version of stateless online workers


تگ 400 تا 401 در تاریخ ۱۳۹۶/۰۷/۱۵

Bugfix: Fix Sanitization to Ignore Some Strings 

By applying sanitization on web request, following to character are
recognized as html tags and so will be ignored (deleted) from request.

Improvement: Do not kill service instances as change service will take care of that

NewFeature: If Not Sub Service Charge Rule Attribute to activate a service only when another service is not activated

Add Group Attr Checker for BW Save Flag

Bugfix :#1639 fix user bw_graph time in mongo branch for user panel

Feature :#1647 add step for ippool percentage in config 

step number added for ippool percentage in config file in INTERFACE
section with interface_ippool_step TAG


تگ 397 تا 400 در تاریخ ۱۳۹۶/۰۷/۰۳

Support Request: #1555 Add User Import Log

Feature: Port auth_event Plugin on C_diameter Branch

Feature: #1460 Send User Events on Message Broker

Bugfix: Include Missing `bm_id` in `applyBwLimit` Method

Improvement: Stop Clearing Terminal on IBSng Startup

Bugfix: #1613 Connection Log Report Problem

Bugfix: #1634 fix connection log report problem 

this problem caused because of new implementation for xss prevention
fixed the place of the code.

Improvement: #1617 Add PubSub to Invoice Profile

Feature: #1553 Add PubSub to Custom Field 

This commit is copy/paste of PR No. 23 which has been done by Morteza
Nourelahi Alamdari on development branch. Because this is a priority
for us in stable branch we create this commit.

Bugfix: #1525 bulk action not found error fix 

it isn't fixed completely yet since ws_workers are more than 1, and
bulk action instance may not have action_id in every ws_worker hence
we face this error. so we sated action_id to $_SESSION so we will
keep it in front side for getting info about it.

Improvement: Add `requests` Module to Python Dependency Packages

Improvement: #1618 Correct Space between Codes

Feature: #1239 Add `online_payment.getAvailableGatewayTypes` Method 

This method have been copy/paste from staging branch because it's been
needed for Wispot project and we can't wait for merging staging branch
to stable one.

Feature: #1611 Add Forgot Password to User Web Panel 

Actually this feature has been implemented before in staging branch by
Saeed Rasooli. Since we are not ready now to merge our staging branch
to stable one, so we have some how copy/paste the changes (with some
cleanup) from staging branch to answer customer request.

Improvement: #1618 Return admin_id in addNewAdmin Method

Bugfix: Fix NTLM Auth Incorrect Variable

Feature: #1572 fix xss security problem and sql injection 

xss csrf and auto complete in form have been fixed .

Add Save BW Usage Condition For Specified Users Only

Bugfix: #1589 TEMPORARY Bug Fix on LDAP Sync Action

Improvement: #1589 Change LDAP Delete/Insert to Update/Insert Action 

Set timeout from "2" to "5" in ldap connection.

Improvement: #1486 Update Bash File Name

Improvement: #1486 Update "return_exception" Docstring

Improvement: #1486 Implement AD Logon Host Feature

Cleanup: UrlDumpDBPool Cleanup

Bugfix: #1515 fix all online graph's time line in user and admin panel 

all the time line issues in both admin panel and user panel for real time graphs have been fixed for branches without mongoDB.

Improvement: #1550 Validate Absolute Expiration Date Input 

Check the epoch time that it won't pass 10 years from now.

Bugfix: #1501 fix group_id and username showing problem in snifflogger

Improvement: Online state keeper refactor and code cleanups to improve performance and correctiveness

Improvement: Ensure we don't leak redis connections in rare cases of pipelines

Cleanup: 0 does not need to be passed to price modifier

Improvement: Made online worker concurrency configurable which is count of worker threads in each online worker. The actual concurrency would be twice this number because threads sending requests to user workers and threads receiving results from user workers are separated

Improvement: Change the order which db handles are used to enable idle checking. It was a FIFO queue and now its LIFO.

Feature: #1533 Exclude parameters from LDAP info in Web Panel 

now we can set the ldap options to show in setting -> advanced -> advanced config -> LDAP PARAM EXCLUDE

NewFeature: Rebin Integration Support

Feature: #1490 Compatible "init_bw_manager" with All Kernel Versions

Fix: Convert getTopConnectionCount Query to MongoDB

Fix: unique_id And unique_id_value Position 

We needed to move unique_id and unique_id_value's position to details
to have it in Report/Connection Logs

Bugfix: Fix 'before_credit' KeyError in PSQL to Mongo Conversion

تگ 397 در تاریخ ۱۳۹۶/۰۶/۰۵

This branch is merge of C_lan_acc into C_diameter_mongo


C_lan_acc Branch

تگ 374 تا 376 در تاریخ ۱۳۹۶/۰۷/۰۳

Support Request: #1555 Add User Import Log

Bugfix: Include Missing `bm_id` in `applyBwLimit` Method

Improvement: #1617 Add PubSub to Invoice Profile

Feature: #1553 Add PubSub to Custom Field 

This commit is copy/paste of PR No. 23 which has been done by Morteza
Nourelahi Alamdari on development branch. Because this is a priority
for us in stable branch we create this commit.

Improvement: Add `requests` Module to Python Dependency Packages

Improvement: #1618 Correct Space between Codes

Feature: #1239 Add `online_payment.getAvailableGatewayTypes` Method 

This method have been copy/paste from staging branch because it's been
needed for Wispot project and we can't wait for merging staging branch
to stable one.

Feature: #1611 Add Forgot Password to User Web Panel 

Actually this feature has been implemented before in staging branch by
Saeed Rasooli. Since we are not ready now to merge our staging branch
to stable one, so we have some how copy/paste the changes (with some
cleanup) from staging branch to answer customer request.

Improvement: #1618 Return admin_id in addNewAdmin Method

Bugfix: Fix NTLM Auth Incorrect Variable

Improvement: #1589 Change LDAP Delete/Insert to Update/Insert Action 

Set timeout from "2" to "5" in ldap connection.

Improvement: #1486 Update Bash File Name

Improvement: #1486 Update "return_exception" Docstring

Improvement: #1486 Implement AD Logon Host Feature

Bugfix: #1515 fix all online graph's time line in user and admin panel 

all the time line issues in both admin panel and user panel for real time graphs have been fixed for branches without mongoDB.

Bugfix: #1501 fix group_id and username showing problem in snifflogger

Feature: #1533 Exclude parameters from LDAP info in Web Panel 

now we can set the ldap options to show in setting -> advanced -> advanced config -> LDAP PARAM EXCLUDE

Feature: #1490 Compatible "init_bw_manager" with All Kernel Versions


تگ 372 تا 374 در تاریخ ۱۳۹۶/۰۵/۲۳

Feature: #1146 Add New Permission "CHANGE USER LIMIT MAC" 

 From now on, only admins with specified permission can modify the
 limit mac attribute of users. This permission has "CHANGE USER
 ATTRIBUTES" permission as it's parent.

Feature: #1462 add failed online reason to user information page 

 if there was a reason for failed online it will be shown on user information page .

Bugfix: #1453 Check All Lines of "ntlm_auth" Result

Bugfix: #1417 Fix Load Balancing Functionality of IPPool Plugin 

 The `ippool_main.getLoader().getIPpoolByID` will return either an
 "IPPool" or "LoadBalancingIPPool" object. In later case the object has
 no `getContainer` method. Instead we use `setIPInPacket` method which
 is available in both objects and has the same functionality.

Feature: #1317 Add ISP Filter to `getListOfCharges` Charge Action

Feature: #1317 Add ISP Filter to `getListOfGroups` Group Action

Bugfix :#1436 change QOS defs propertis to accept string input

Bugfix :#1424 real time bw graph timeline problem 

 time line problem solved using utc time guide.

Improvement: don't log wrong password

Bugfix :#1426 top visited urls problem 

 direction to the urls has been edited .

Bugfix: #1367 search_expired_user.php check_all_users problem 

 multi select problem for users has been handled.

Bugfix: #1408 Port call_id_blacklist plugin to C branch

Bugfix: #1333 Fix Memory Leak on LDAP Sync Action 

 According to get several logs of different situations from server and
 analysing them, this problem has been occured on new thread (event.AddEvent).
 In "_startSync" method after fetching list of users, it set "self._startSync"
 in event for a period of time and problem starts from here. So to fix this issue,
 instead of spawing new threads for each domains on "event" class, we keep continue to
 use domains threads which created on IBS start up.

Fix: Show Exact Value for ISP's on Online Users

Feature : #1361 change the sub service qos name to limited drop down 

 change  the sub service qos name to limited drop down options .

Feature : #1360 edit charge priority from 1-10 to 1-50 

 edit charge priority to 1-50 and it's backend error and validation rules.

تگ 371 تا 372 در تاریخ ۱۳۹۶/۰۳/۲۴

- Bugfix: Fix CSV Report of Web Analyser

- Bugfix: #1339 Load Invoice Main in Online Worker

- Bugfix : #1300 delete realtime request log interface 

 real time request log  deleted  from interface because the source code of this link  had been deleted before .

- Bugfix: #1339 Load Invoice Main in Online Worker

- Feature : #1223 Modify 'urldump' sniffer application to fetch 'ip_map' data directly from IBSng

- Bugfix: Fix LDAP Permissions

- Bugfix: #1278 Fix Account Not Started Counter 

 When "account not started" counter got negative value,
 IBSng web panel shown "0*" value that means negative number.
 The main reason for this issue was when a user had several sub services,
 it tried to decrease counter per each of them (when their requests received),
 while it increased counter once on authentication action. So to solve this issue,
 we handled __decreamentNotStartedCounter method which check loaded user's
 instance that must be Master to run decrease action.
 Example diagram for this issue:
  Step1: Authentication +1   << Correct
  Step2: Master         -1   << Correct
  Step3: Free-Site      -1   << Incorrect
  StepN: SubServiceN    -1   << Incorrect
  StepX: SubServiceX    -1   << Incorrect

- BugFix : #1263 fix urldump does'nt showing group name in web analyzer log 

 this bug fixed in the interface.
 it also fixed in real time web analyzer page.

- Bugfix: #1292 Fix Bulk Kill Action in Case of Choosing More Than One Group 

 When user chooses more than one group/isp/etc in Search User panel and
 selects bulk kill action it raises and error. It was because of
 invalid format of passing parameters.

- Cleanup: We Don't Have `web_analyzer_log` Table Any More 

 Also raised an error when we wanted to delete a user on deleting web
 analyzer logs, because there is no such table available any more

- Bugfix: Fix Minor Typo Error

- Bugfix: #1235 Fix Minor Invoicing DB Bug 

 Column reference to 'pi_issue_date' was ambiguous!

- Bugfix: #1228 Fix Max Time Usage Charge From Custom Field 

 Resolving 'max_rule_time_usage' charg rule to be read from user
 custome field (e.g: 'F:time') haven't been handled.

- Bugfix: #1215 Fix Timezone Issue in Online Graph


تگ 369 تا 371 در تاریخ ۱۳۹۵/۱۲/۱۰

- Bugfix: Fix Active Leaves 

 The `getActiveLeaves` method must be called in 'MAIN' process.

- BugFix: We should not add internal_relogin flag to user attributes

- BugFix: Various Internal Relogin fixes

- Improvement: Make sure listen loop won't exit on an abnormal exception

- BugFix: Setting max rule traffic usage value as a float in custom fields would have break max rule traffic plugin

- Improvement: Disable Partitioning Check Contraints 

 This is done because of some efficiency issues

- Improvement: Disable adding check constraint to partitioned tables

- BugFix: We should ignore negative threshold when QV is passed to cisco isg

- BugFix: A mid session reauthorize after a restart will create an instance without start accounting

- BugFix: Do not panic if charge rule is None during stop session

- BugFix: Not saving user after logout will lead to inconsistent user object

- BugFix: Take credit limit into account when calculating remaining bytes. This fixes postpaid user problem with isg prepaid services

- Improvement: Kill max delay default value was too large. Set it to 2 hours by default

- Improvement: Make auth rate control optional and turn it off by default

- Bugfix: #1512 Fix `int` Convertion of `float` String

- Bugfix: #1449 Fix `users.group_id` Bug 

 Fixed `getDBColName` in a way which returns the part after the dot,
 e.g 'usersgroup_id' => 'group_id'

- Bugfix: #1449 Fix Ambiguous Fields `user_id` and `group_id` in Management Summary Report 

 After adding two columns `user_id` and `group_id` to `connection_log`
 table it created a conflict with `users` table on in management
 summary queries; in this fix we change SQL queries to exactly specify
 which fields we mean by `user_id` and `group_id`: `users.user_id`,
 `users.group_id`, etc.

- Bugfix: add missing function call on commit of periodic acc plugin

- Bugfix: #1503 Fix MAC Extraction From RAS of ZTE Type 

 ZTE model rases change the format of user mac address in recent
 framework update, e.g:
 old format: `78d99fd34e4b`
 new format: `78:d9:9f:d3:4e:4b`

- Improvement: Few logging for radius process

- BugFix: Reauthorize bytes were deducted twice from remaining bytes

- BugFix: Do not start accounting on reauthorize. Removes lots of nagging about online dic not found in error log

- Improvement: Make sure we never return a quota of 0 in case of a successful reauthorize

- Improvement: Run internal relogin with same priority as auth. Long queues might cause WS request to return while task has not been done yet

- BugFix: reauthorize_used_bytes should be used bytes from last update and not from start of session. Fixed premature QV0 while user still has credit.

- Bugfix: don't assign ip to sub service

- Bugfix: skip users without ignore_ras

- Feature: Add support for cisco_isg_max_reauthorize_mbytes and cisco_isg_max_reauthorize_minutes attributes to force reauthorize after a certain threshold

- Feature: Multiple prepaid services per account support

- BugFix: Max Rule Traffic Usage were not working correctly with multiple rules sharing one value (and custom field) and multiple instances using them

- BugFix: Onlines table in single user info page were became messed up after first ajax based refresh

- BugFix: Multi login did not work with Multi Service rases other than ISG

- Bugfix: #1452 Fix/Update Deprecated Links in Graph Menu 

 The following links are deprecated and removed:
 All Realtime Graph
 Internet Realtime Graph
 Voip Realtime Graph
 BW Realtime Graph  
 Also updated modified links

- Improvement: Failure to change sub services (ex. failed COA) should be detected. Also resync services after internal relogin to make sure we have correct services

- Bugfix: #1136 Fix Name of Import File in Import User Menu 

 Note: after changing the log process to pass the logging mechanism to
 python syslog module, user_import logs won't be written to a local
 file which will cause to "View Import Log" link on "Import User" page
 after uploading CSV file won't work. this can be fixed by adding new
 log type (aside `QueueLogger` and `DirectLogger`) to IBS logger
 module, but this is not a priority now; look at the comments on
 changed files for more info.

- BugFix: Re-Onlined users might have get failed status due to left over from previous sessions

- BugFix: Radius Attributes with integer values were not working in charge rule attribute radius

- Improvement: Allow Acct-Interim-Interval to be passed to failed users

- Improvement: Exception handling for internal relogin

- BugFix: Removed Extra print

- Bugfix: Incorrect call to getUserAttrs caused exception

- BugFix: Some initial variables where missing in configuration file and advanced configuration

- BugFix: Undefined session raised unnecessary exception

- Improvement: API cleanups for cisco isg rases

- Bugfix: cast ignore ras ids to int

- Bugfix: read min/max event time directly from defs

- Bugfix: fix group usages of connection analysis

- Improvement: Remove `price` Filter From User ID Attribute

- Feature: Add User ID Attribute to Search User Page

- BugFix: Threshold Bytes/Seconds did not trigger internal reauthorize due to bad merge

- BugFix: Second username change fail in case of internal relogin

- Improvement: get method for SNMP client

- Feature: optionally do autoRenew on abs expired user

- Bugfix: log deposit_type in deposit change

- Cleanup: clean md5 response

تگ 359 تا 360 در تاریخ ۱۳۹۵/۱۱/۱۶

- Bugfix: add removed ad user info in comment tab

- Bugfix: affect search base for paged searches


C_lan_mongo Branch

تگ 371 تا 372 در تاریخ ۱۳۹۶/۰۳/۲۷

- Bugfix: Fix CSV Report of Web Analyser

- Bugfix: #1339 Load Invoice Main in Online Worker

- Bugfix : #1300 delete realtime request log interface 

 real time request log  deleted  from interface because the source code of this link  had been deleted before .

- Bugfix: #1339 Load Invoice Main in Online Worker

- Feature: #1223 Modify 'urldump' sniffer application to fetch 'ip_map' data directly from IBSng

- Bugfix: Fix LDAP Permissions

- Bugfix: #1278 Fix Account Not Started Counter 

 When "account not started" counter got negative value,
 IBSng web panel shown "0*" value that means negative number.
 The main reason for this issue was when a user had several sub services,
 it tried to decrease counter per each of them (when their requests received),
 while it increased counter once on authentication action. So to solve this issue,
 we handled __decreamentNotStartedCounter method which check loaded user's
 instance that must be Master to run decrease action.
 Example diagram for this issue:
  Step1: Authentication +1   << Correct
  Step2: Master         -1   << Correct
  Step3: Free-Site      -1   << Incorrect
  StepN: SubServiceN    -1   << Incorrect
  StepX: SubServiceX    -1   << Incorrect

- BugFix : #1263 fix urldump does'nt showing group name in web analyzer log 

 this bug fixed in the interface.
 it also fixed in real time web analyzer page.

- Bugfix: #1292 Fix Bulk Kill Action in Case of Choosing More Than One Group 

 When user chooses more than one group/isp/etc in Search User panel and
 selects bulk kill action it raises and error. It was because of
 invalid format of passing parameters.

- Improvement: #1186 Run mongo index in thread

- Improvement: #1186 Run mongo index in thread

- Cleanup: We Don't Have `web_analyzer_log` Table Any More 

 Also raised an error when we wanted to delete a user on deleting web
 analyzer logs, because there is no such table available any more

- Bugfix: #1235 Fix Minor Invoicing DB Bug

- Bugfix: Fix Minor Typo Error

- Bugfix: #1235 Fix Minor Invoicing DB Bug 

 Column reference to 'pi_issue_date' was ambiguous!

- Bugfix: Correct Delete Query of Redundant Partial Log Records

- Improvement: Read From Config File Whether to Write on PSQL or Not 

 Until now we read `defs.PSQL_LOG_CONNECTIONS` value to define whether
 we should write connection logs into PostgreSQL or not. With this
 commit value of `connection_log_use_psql` in `DATABASE_MONGO` section
 of IBSng config file defines our behaviour.

- Bugfix: #1231 Fix Total Values of User Connection Logs Panel 

 Before migrating to MongoDB we used "1" to indicate whether to show
 total values in connection log report or not, but after migration we
 used "On"!!, this has been handled in this commit to work with both of
 them.

- Bugfix: #1228 Fix Max Time Usage Charge From Custom Field

- Bugfix: #1228 Fix Max Time Usage Charge From Custom Field 

 Resolving 'max_rule_time_usage' charg rule to be read from user
 custome field (e.g: 'F:time') haven't been handled.

- Bugfix: #1201 Fix Slow Start of IBSng and Multiple Partial Log

- Improvement: #1154 Add More Log in Case of Bulk Write Error

- Bugfix: #1215 Fix Timezone Issue in Online Graph

- Bugfix: #1201 Fix Slow Start of IBSng and Multiple Partial Log 

 It has been seen that the number of online user doesn't match the
 records count on partial data table, it will result in slow start of
 IBSng (in recover partial log phase). This commit will fix this issue
 and makes some improvement on writing partial log data.

- Improvement: Optimized partial log recovery for postgresql. Not exists queries are much more efficient for large inner queries with index

- BugFix: Internal relogin cleanup and bug fixes

- Improvement: Increased default max thresholds to decrease the load on ras and ibs

- Bugfix: #1474 remove please wait in report pages 

 removing blockui  javascrpit code which caused please  wait deadlock in report pages


C_lan_acc_staging Branch

تگ 371 تا 372 در تاریخ ۱۳۹۶/۰۳/۲۷

- Bugfix: Fix CSV Report of Web Analyser

- Bugfix: #1339 Load Invoice Main in Online Worker

- Bugfix : #1300 delete realtime request log interface 

 real time request log  deleted  from interface because the source code of this link  had been deleted before .

- Bugfix: #1339 Load Invoice Main in Online Worker

- Feature : #1223 Modify 'urldump' sniffer application to fetch 'ip_map' data directly from IBSng

- Bugfix: Fix LDAP Permissions

- Bugfix: #1278 Fix Account Not Started Counter 

 When "account not started" counter got negative value,
 IBSng web panel shown "0*" value that means negative number.
 The main reason for this issue was when a user had several sub services,
 it tried to decrease counter per each of them (when their requests received),
 while it increased counter once on authentication action. So to solve this issue,
 we handled __decreamentNotStartedCounter method which check loaded user's
 instance that must be Master to run decrease action.
 Example diagram for this issue:
  Step1: Authentication +1   << Correct
  Step2: Master         -1   << Correct
  Step3: Free-Site      -1   << Incorrect
  StepN: SubServiceN    -1   << Incorrect
  StepX: SubServiceX    -1   << Incorrect

- BugFix : #1263 fix urldump does'nt showing group name in web analyzer log 

 this bug fixed in the interface.
 it also fixed in real time web analyzer page.

- BugFix: reloadUser without wait for lock caused infinite loop

- BugFix: Online User Storage must be flushed during IBSng start up

- Cleanup: Cleaned up check online method

- BugFix: Fixed reloadUser might incorrectly run while lock has not been acquired

- BugFix: Redis might incorrectly reset pubsub in busy situations

- BugFix: Credit Limit was displayed incorrectly

- BugFix: Mutable object should not be passed as default argument

- BugFix: Fix Relogin User Issue On Update Custom Field

- Bugfix: #1292 Fix Bulk Kill Action in Case of Choosing More Than One Group 

 When user chooses more than one group/isp/etc in Search User panel and
 selects bulk kill action it raises and error. It was because of
 invalid format of passing parameters.

- Bugfix: #1283 Add Back Missing LDAP SOAP Functions 

 Some of LDAP SOAP functions have been deleted mistakenly in previous
 merges.

- BugFix: mp observer change in isp loader caused isp states to reset

- BugFix: unload/load instead of reload causes race condition where isp object is no longer available

- Cleanup: We Don't Have `web_analyzer_log` Table Any More 

 Also raised an error when we wanted to delete a user on deleting web
 analyzer logs, because there is no such table available any more

- BugFix: Online user storage were using incorrect redis db

- Bugfix: #1235 Fix Minor Invoicing DB Bug 

 Column reference to 'pi_issue_date' was ambiguous!

- Cleanup: Remove Unused Method `__renewMustResetCredit`

- BugFix: Kairos might be initialized incorrectly because system redis library was imported instead of local one

- BugFix: Fixed Typo which cause isp deposit role back not to work correctly

- BugFix: __internetReauthorizeCheckForIncorrectSubService was not working due to typo

- BugFix: Fix memory leak of web service process by replacing cyclic-reference-creator collections.OrderedDict with odict.odict in kairos library

- BugFix: Consume ISP deposit was broken after Internal Relogin. Fix to always reload ISP and log after consuming deposit.

- Bugfix: #1228 Fix Max Time Usage Charge From Custom Field 

 Resolving 'max_rule_time_usage' charg rule to be read from user
 custome field (e.g: 'F:time') haven't been handled.

- BugFix: Duplicate detector was not caching reply packets and never replay them

- Bugfix: Remove Remaining Partitioning Check Code 

 Since we don't check for partitioning capability of PostgreSQL any
 more (see recent commits) we don't need this any more.

- Bugfix: #1215 Fix Timezone Issue in Online Graph

- Bugfix: Remove Calling Deleted Module

- Improvement: Log next event and kill dict in can stay event of log console

- BugFix: Prevent maximum threshold minutes might have caused avalanche situtation, causing cisco ISG bras to overload

- BugFix: Use Acct-Session-Id in kick user via coa instead of Cisco-Account-Info

- BugFix: Internal relogin cleanup and bug fixes

- Improvement: Increased default max thresholds to decrease the load on ras and ibs

- Cleanup: Change RAS Name 'Wireless' to '4ipnet'

- Cleanup: SOAP Cleanup

- Feature: add isp mapping from LDAP

- Feature: add send_lock_users capability to notification rules

- Improvement: use whenchanged attribute in ldap query


تگ 360 تا 371 در تاریخ ۱۳۹۵/۱۲/۱۰

- Bugfix: Fix Active Leaves 

 The `getActiveLeaves` method must be called in 'MAIN' process.

- BugFix: We should not add internal_relogin flag to user attributes

- BugFix: Various Internal Relogin fixes

- Improvement: Make sure listen loop won't exit on an abnormal exception

- BugFix: Setting max rule traffic usage value as a float in custom fields would have break max rule traffic plugin

- Improvement: Disable Partitioning Check Contraints 

 This is done because of some efficiency issues

- Improvement: Disable adding check constraint to partitioned tables

- BugFix: We should ignore negative threshold when QV is passed to cisco isg

- BugFix: A mid session reauthorize after a restart will create an instance without start accounting

- BugFix: Do not panic if charge rule is None during stop session

- BugFix: Not saving user after logout will lead to inconsistent user object

- BugFix: Take credit limit into account when calculating remaining bytes. This fixes postpaid user problem with isg prepaid services

- Improvement: Kill max delay default value was too large. Set it to 2 hours by default

- Improvement: Make auth rate control optional and turn it off by default

- Bugfix: #1512 Fix `int` Convertion of `float` String

- Bugfix: #1449 Fix `users.group_id` Bug 

 Fixed `getDBColName` in a way which returns the part after the dot,
 e.g 'usersgroup_id' => 'group_id'

- Bugfix: #1449 Fix Ambiguous Fields `user_id` and `group_id` in Management Summary Report 

 After adding two columns `user_id` and `group_id` to `connection_log`
 table it created a conflict with `users` table on in management
 summary queries; in this fix we change SQL queries to exactly specify
 which fields we mean by `user_id` and `group_id`: `users.user_id`,
 `users.group_id`, etc.

- Bugfix: add missing function call on commit of periodic acc plugin

- Bugfix: #1503 Fix MAC Extraction From RAS of ZTE Type 

 ZTE model rases change the format of user mac address in recent
 framework update, e.g:
 old format: `78d99fd34e4b`
 new format: `78:d9:9f:d3:4e:4b`

- Improvement: Few logging for radius process

- BugFix: Reauthorize bytes were deducted twice from remaining bytes

- BugFix: Do not start accounting on reauthorize. Removes lots of nagging about online dic not found in error log

- Improvement: Make sure we never return a quota of 0 in case of a successful reauthorize

- Improvement: Run internal relogin with same priority as auth. Long queues might cause WS request to return while task has not been done yet

- BugFix: reauthorize_used_bytes should be used bytes from last update and not from start of session. Fixed premature QV0 while user still has credit.

- Bugfix: don't assign ip to sub service

- Bugfix: skip users without ignore_ras

- Feature: Add support for cisco_isg_max_reauthorize_mbytes and cisco_isg_max_reauthorize_minutes attributes to force reauthorize after a certain threshold

- Feature: Multiple prepaid services per account support

- BugFix: Max Rule Traffic Usage were not working correctly with multiple rules sharing one value (and custom field) and multiple instances using them

- BugFix: Onlines table in single user info page were became messed up after first ajax based refresh

- BugFix: Multi login did not work with Multi Service rases other than ISG

- Bugfix: #1452 Fix/Update Deprecated Links in Graph Menu 

 The following links are deprecated and removed:
 All Realtime Graph
 Internet Realtime Graph
 Voip Realtime Graph
 BW Realtime Graph  
 Also updated modified links

- Improvement: Failure to change sub services (ex. failed COA) should be detected. Also resync services after internal relogin to make sure we have correct services

- Bugfix: #1136 Fix Name of Import File in Import User Menu 

 Note: after changing the log process to pass the logging mechanism to
 python syslog module, user_import logs won't be written to a local
 file which will cause to "View Import Log" link on "Import User" page
 after uploading CSV file won't work. this can be fixed by adding new
 log type (aside `QueueLogger` and `DirectLogger`) to IBS logger
 module, but this is not a priority now; look at the comments on
 changed files for more info.

- BugFix: Re-Onlined users might have get failed status due to left over from previous sessions

- BugFix: Radius Attributes with integer values were not working in charge rule attribute radius

- Improvement: Allow Acct-Interim-Interval to be passed to failed users

- Improvement: Exception handling for internal relogin

- BugFix: Removed Extra print

- Bugfix: Incorrect call to getUserAttrs caused exception

- BugFix: Some initial variables where missing in configuration file and advanced configuration

- BugFix: Undefined session raised unnecessary exception

- Improvement: API cleanups for cisco isg rases

- Bugfix: cast ignore ras ids to int

- Bugfix: read min/max event time directly from defs

- Bugfix: fix group usages of connection analysis

- Improvement: Remove `price` Filter From User ID Attribute

- Feature: Add User ID Attribute to Search User Page

- BugFix: Threshold Bytes/Seconds did not trigger internal reauthorize due to bad merge

- BugFix: Second username change fail in case of internal relogin

- Improvement: get method for SNMP client

- Feature: optionally do autoRenew on abs expired user

- Bugfix: log deposit_type in deposit change

- Cleanup: clean md5 response

تگ 359 تا 360 در تاریخ ۱۳۹۵/۱۱/۱۶

- Bugfix: add removed ad user info in comment tab

- Bugfix: affect search base for paged searches

برگرفته از «http://wiki.parspooyesh.com/index.php?title=%D8%A2%D8%AE%D8%B1%DB%8C%D9%86_%D8%AA%D8%BA%DB%8C%DB%8C%D8%B1%D8%A7%D8%AA_IBSng&oldid=2814»
ابزارهای شخصی

گویش‌ها
فضاهای نام
عملکردها
گشتن
جعبه‌ابزار